UWS academic responds to BBC Scotland criticism of indy news study


  Reply, by Dr John Robertson, to the BBC (Ian Small, Head of Public Policy & Corporate Affairs) response to his interim research report:  Fairness in the First Year? BBC and ITV coverage of the Scottish Referendum campaign from September 2012 to September 2013.
My research is the result of a powerful philosophical commitment over decades to ‘grounded theory’; a form of research almost monastic in its restraint and in its potential to allow the data to tell the story that is there in the data and less so the story that lies in the researcher’s baggage. None of the team is a nationalist.

I am a socialist, pacifist and feminist. I had no agenda to pursue other than genuine curiosity to explore the patterns of journalistic behaviour in a process of wide contemporary interest. It is my usual preference to study thought control in the context of military interventions, mega sporting events, surveillance and the environment – see my CV.

Mr Small uses the term ‘we’ throughout so I must assume his comments have been approved at the highest level in the BBC. I should stress that my report does not represent the corporate views of UWS.

I’ll move on quite quickly to a more detailed and specific rejoinder but first and briefly, a little background is necessary


The BBC responses came with a covering email including the three comments below.

As noted in our earlier correspondence, we would be very interested in seeing the raw data which underpinned your report and we wonder if you might be willing to share that with us.

No. Would you have asked to see the raw data if the report had been favourable to the BBC? Have you ever asked another researcher for such? Indeed, aren’t the broadcasts you own, the raw data?

Further, given the responsibilities you remind us of in Appendix 1, what have you actually done to monitor the coverage. Do you have an internal report along lines similar to my research? How do your findings compare? Can I see the raw data?

We would, of course, be very happy to talk to you about your report and our interpretation of it.

Report on it (Reporting Scotland). Let your experts comment then let me reply to them. Post all the documents on your website so that licence payers may read them carefully. It is not appropriate for media managers to suppress research they disagree with and then mount a campaign to discredit it while at the same time ignoring its presence online.

The reason we have copied this email to the Principal, Professor Craig Mahoney (as we have all of the correspondence we have sent to you), is because we believe the report, with which we have serious concerns, has the potential to impact on the corporate reputation of both institutions.

This sails very close to bullying of the kind we might expect in a less democratic country. I’m unsure about your limiting of reputation to the corporate. Universities are so much more than that. I’d be more worried if I thought it might damage our wider reputation in the public sphere or with potential applicants to study with us. I suspect, based on the massive online debate which you ignore, it will have the opposite effect.

I have not copied this to your head of institution.

BBC Scotland response to the University of West of Scotland report:  Fairness in the First Year? BBC and ITV coverage of the Scottish Referendum campaign from September 2012 to September 2013, by Dr John Robertson.

The BBC has a number of significant concerns about the contents of the UWS Report Fairness in the First Year? BBC and ITV coverage of the Scottish Referendum campaign from September 2012 to September 2013.

These focus, primarily, around elements of the research methodology, factual accuracy, interpretation and conclusions and the language used within the report itself.

As a consequence, we question the fundamental validity of this report and, in particular, the conclusions which it reaches.  A detailed critique follows.

Social and political research, however well-grounded in good methodology, retains a level of subjectivity, especially in the coding categories and in their application. I make no claim for absolute truth; only a level of objectivity somewhat above that attainable by journalists and corporate affairs officers.

There are factual errors in some of the examples given. Large scale research transcribed and analysed over some time by a team of researchers will find such errors introduced. I do regret these as their import can be inflated. In this case, all those reported are not substantive and are inconsequential in the evaluation of the research outcomes.

In Summary, our concerns are around:

See detailed responses for these below. They are in every case misplaced or exaggerated concerns.

Fairness – the report concludes that BBC Scotland TV coverage of the Referendum has not been fair or balanced.  However the report consistently fails to support its contentions with factually accurate evidence.

Accuracy – we would contest the interpretation of such evidence as is presented within the report, relating to Reporting Scotland (evidence largely presented by reference to examples of output). There are several substantive factual inaccuracies within the references it makes to Reporting Scotland news output.

Interpretation – we are concerned about the ‘coding’ of stories – a number of the examples cited within the report, which claim to refer to referendum stories, are not referendum stories.

Methodology – the report appears to be based on a numerical assessment of data, analysed in crude quantitative terms, following readings of transcripts of broadcasts. We would question the validity of conclusions based on such an approach, particularly given the lack of detail and the factual errors within the report. 

Definitions – the report presents its findings relative to a number of terms which, apart from on one occasion (relative to what it deems to be a ‘statement’) it singularly fails to define – ‘fairness’, ‘insulting language’ etc. There is no indication anywhere in the report as to what is meant by/should be understood by such terminology, by whose definition broadcast contents were to be considered as ‘fair’ or ‘insulting’ or whether any account was taken of what the BBC’s own Editorial Guidelines or the Ofcom Broadcasting Code have to say in this respect.

Conclusions – the report concludes that the report authors have “evidence of coverage which seems likely to have damaged the Yes campaign.” There is no evidence whatsoever, as contained within the report, that supports this contention. It is no more than an assumption, based on the report’s findings which, themselves, we contest.

Section 1


a) ‘Fairness’

The report clearly takes ‘fairness’ to be its focus. However at no point does the author make clear what he means by this term, how he interprets what he perceives to be ‘fair’ or ‘unfair’ or the criteria by which readers should understand his interpretation. He does not indicate if or how his interpretation, whatever that might be, aligns or differs from the BBC’s (in respect of which, the BBC Editorial Guidelines [1] lay down detailed criteria to ensure due accuracy and impartiality (or ‘fairness’) across its output).

We would note that the BBC Editorial Guidelines state: ‘Due impartiality is often more than a simple matter of ”balance’ between opposing viewpoints.’  Further details of the relevant section of the BBC’s Editorial Guidelines  are incorporated within Appendix 1 (page 17).

Ofcom defines fairness as below [2]:

Principle: To ensure that broadcasters avoid unjust or unfair treatment of individuals or organisations in programmes.

7.1 Broadcasters must avoid unjust or unfair treatment of individuals or organisations in programmes.

It appears that the report’s understanding of ‘fairness’ differs from the definition of ‘fairness’ used by the broadcasting industry regulator (and thus by UK broadcasters and by the BBC). However, like much of the terminology applied in this report, the term ‘fairness’ remains undefined throughout.

The term ‘Fairness’ does appear in the title of the report. It is, however, not featured in any of the nine research questions which do focus the research. A series of more objective terms such as ‘how prevalent’ or ‘what was the relative balance’ are used to enable the subsequent data collection and coding to proceed as objectively as is possible in textual analysis. No claim is made for an absolute objectivity in this process. To do such would be ridiculous. As for ‘fairness’ there is no universally accepted definition. I agree, however, as you argue above, that it is more than just about balance. That’s why there are as many as nine research questions and not just one on balance. I propose that my cluster of criteria, manifest in the research questions, offers a workable expression of fairness in the context of news reporting. Not perfect I’m sure but worthy of ongoing debate in the public sphere.

Further, as pointed out in many emails to me, it could be argued that I have been more than fair to BBC/ITV in my methodology of only researching that which was broadcast. I did not survey relevant stories emerging elsewhere which were not selected for reporting. There could easily be bias there. I’m told there is. To research this area was, however, too unavoidably subjective a field for me.

b) Objectivity

The report notes that the survey period ran “from 17th September 2012 to 18th September 2013 including every evening (6-7PM) broadcast by BBC1, Reporting Scotland, ITV and STV, in that period.”   We take this to mean, though it does not clearly say it, that it was exclusively the news reports within that news hour that were examined. Our critique will therefore focus on contentions made about those news broadcasts.

The preface states the researchers “sought to disengage themselves from the surrounding debate, in extended newspaper articles or TV debates”. How was this achieved over the course of an entire year? What steps were taken to ensure such disengagement and how was the effectiveness of such an approach measured to ensure, as the report indicates, “as objective an assessment as..possible” ? Did this mean the research team did not read newspapers, talk to friends, go online or watch other TV output?  In terms of external engagement, it is clear that the press is used as a reference point within the report, which, on page 7, quotes from the Daily Mail of 9/11/12 and refers to a wider comparative study of press stories: “No balancing cases were reported of a flow in the other direction, though such did appear in the popular press.”

Further, we would be interested to understand what control mechanisms were used to ensure the validity of the research and to assess the accuracy of its findings, particularly given the multiple variables therein – 9 questions and 16 coded descriptions applied to 730 hours of content analysed across 2 channels and 4 programmes.  We would also be interested to know what processes were put in place to guarantee the standardisation of approach and impartiality across the number of staff, students and others involved in the transcription and process of analysis over one year.  Given that the amount and complexity of coverage will increase in the coming year, we query whether the approach is scalable and what % of accuracy will result.  We assume that any further publication will be supported by a detailed methodology.

Your first point above is confusing. What do you mean by exclusively the news reports? Yes, I think.

The second point is expressed in quite patronising terms. However, it if helps, here is our plea. I (lead researcher) did not watch or read, and still do not, texts relating to the Scottish independence debate. My teaching and other research interests are concerned with the globalisation of news media and events in places such as Russia or Venezuela. My other coders, likewise, are not much interested in the independence referendum and are pursuing research in other areas. The Daily Mail reference was provided by a proof reader who, it seems, reads it!

Again, I dare you to ask one of your regularly appearing professors a question like that in para three above. Again, here you are. The research is not based on a small sample, as is common in surveys often reported on TV without critique but is based on one whole year and every evening from 6-7 pm (at other times on weekends) on two channels. The coding which led to the evidence of bias emerged from a grounded theory/ phenomenological approach which allows the data to speak. The final coding is the product of two phases, through all the data, of coding by the lead researcher and subsequent moderation by three others (recently retired staff and PhD students).

The first phase resulted in evidence of bias more damaging to the BBC and STV. In the second phase, the lead researcher allocated statements with more subtle or nuanced undermining of the Yes campaign to the general or descriptive category. Coding of human language cannot be utterly objective but the team has done more than most in an effort to be as objective as can be. The lead researcher has carried out similar studies, in terms of methodology, over many years, and takes pride their publication in the best international journals.

c) Hours

The report notes that it covers the 6.00PM – 7.00PM news hour over the course of a year – “A total, therefore, of approximately 730 hours, minus advertising breaks in ITV and STV broadcasts, was watched, transcribed and coded.”

We would like better to understand how this hours figure was constructed.  Was it 365 (days) x 2 (hours) = 730 hours?  

The 6.00PM – 7.00PM news hour only operates during weekdays, of which there are 260 over the course of 52 weeks. That, therefore, only amounts to around 520 hours of weekday news across the two channels between 6.00PM and 7.00PM, not 730 hours (260 x 2 – and this does not take account of shorter news broadcasts on public holidays).

Even if the report were to include weekend bulletins between 6.00PM – 7.00PM, which are of significantly shorter duration (though the report does not seem to include these), that would offer only another 80 – 90 hours per annum.

Still, one whole year of peak news broadcasts and all references to issues affecting the independence debate. ‘Approximately 730 hours’ is inaccurate. It was 365 days but less than 2 hours at the weekend. So approximately there were 620 hours plus those allocated to extended broadcasts after key events. Do you wish to argue that this affects anything at all?

d) Questions

The report then outlines a number of questions which it says emerged from readings of transcripts of news items.  This in itself is worthy of question – are the research findings as they appear in this report based on consideration of written transcriptions rather than on first-hand assessment of the actual broadcasts and of their audio and video content, which then fed into this report?

The lead researcher watched, recorded, transcribed and coded using only the transcripts.

What, if any, account was taken of tone, of the visual use of graphics within broadcasts to identify contributors and the source of statistics, etc?  These, we believe, are important questions because the report, as later illustrated (herein), contains a significant number of factual inaccuracies and seems to have drawn a series of conclusions based on a misreading of the contents of a number of news items.

Tone, graphics – far too subjective for this kind of research.

To look at the questions themselves:

(i) How prevalent were referendum topics in the first year of the campaigns?
To assess ‘prevalence’, which is a relative term, the total number of items covered in the news broadcast would have to be known (as would the position of referendum items within each day’s running order, the length of items, etc).

Are you confusing prevalence with incidence? I think my readers understand. Nitpicking?

Likewise, there would have to be a clear understanding of what was a ‘referendum topic’ and what constituted the reporting of everyday political issues, including those relating to devolved powers. The report does not indicate the total number of such items, how each was interpreted and coded or any other information relating to them; consequently the reader cannot assess the validity of the analysis from the evidence provided.

A ‘referendum topic’ would be anything we thought might affect viewers thinking about Scottish independence. Subjective I agree but done so as to ensure relevant elements of the debate were not ignored.

As such, a number of the Reporting Scotland examples cited herein as referendum stories (or ‘referendum topics’) had nothing to do with the referendum.

Nothing? Nothing? Let’s see.

I agree that this is a subjective process where some will agree with our choices and others will disagree. I suggest the desire to narrow referendum topics to only those explicitly mentioning it can be an ideologically driven perspective with the aim of limiting criticism of coverage.

(ii) What was the relative balance of statements given to the views of Yes and No, representatives, arguments and evidence?

We take this to mean ‘Yes and No representatives’ arguments and evidence’.

Including the repetition and paraphrasing of these by reporters and the statements by ‘experts’ whose views were clearly supportive on one side or the other/

What is meant here by ‘relative balance’ – how has it been defined, ‘relative’ to what?

Simple ratio of Yes:No

(iii) What was the relative balance of offensive statements made to Yes and No campaigners and broadcast?

What is meant by ‘offensive’? No definition or criteria are offered to allow readers to understand what is meant by this. Does this include, for example, remarks made in debate in Holyrood?  In such circumstances, it is the duty of the Presiding Officer to determine what is or is not ‘offensive’.  Does the research reflect such determinations or have alternative criteria subjectively been applied by the research team? If so, what are they?   What account was taken, if any, of the definitions of ‘offence’ as recognised within the BBC’s Editorial Guidelines and within the Broadcasting Code published by Ofcom, the broadcasting industry regulator?

In the event, almost always comments by the Labour, Conservative and LibDem leaders accusing the First Minister of dishonesty or other personal weaknesses and left in the broadcasts. It is not good enough to blame the Presiding Officer. Why were these insults broadcast?

Our coverage of verbal exchanges within the Chamber is determined entirely on editorial grounds. 

Meaning? I’m sure your guidelines would allow editing out such comments.

(iv) What forms of evidence dominated the discourse – economic, political, social?

Analysis of such issues may be interesting but there is no clarity offered within the report as to the relevance of each (or all) to an assessment of ‘fairness’ in news coverage of the referendum.

Yes, I should have made more of the agenda setting of editors in characterising the Scottish public as concerned only about economic matters. I’d like to see the evidence of this other than in the discourse among journalists and a small number of small sample surveys. I see signs elsewhere of a wider discourse, around democracy, compassion and rights, absent in the news broadcasts researched here. The narrowing of debate to shallow materialist concerns can be seen as ideologically driven efforts to strip the Yes campaign of opportunities to debate. I’m reminded of my days as a young football fan when my small home team would narrow the pitch for visits by the big clubs thus compressing the play and stifling creativity. I have, of course no objective evidence for this.


The report then outlines a number of ‘coding categories’ used to analyse the data (though the report offers no appendices, footnotes, bibliography or references to allow further interrogation of whatever data was analysed or of the criteria applied in the process of that analysis). 

The coding categories, which emerged from pilot exercises, were the criteria used in the process. Really not sure what the above point is getting at.

Pro/Ind/Sci/Acad Evidence
Statements which made use of academic, scientific or ‘independent’ evidence to support the pro-independence or Yes campaign

Anti-/Ind/Sci/Acad Evidence
Statements which made use of academic, scientific or ‘independent’ evidence to support the anti-independence or No campaign

No indication is given here of who has ‘made use of’ any such statements.  Is the suggestion that BBC Scotland has ‘made use of’ such statements in its reporting?  If so, is the suggestion then, as it would seem to be, that such use was made by BBC “in support” of either campaign? 

This verges on a paranoid reading. Clearly the statements are made by proponents and sometimes repeated by reporters. Where do you find evidence for the last statement above?

If this is the case, this reveals a fundamental misunderstanding of how News, and in particular BBC News, operates.  As an impartial news broadcaster, it is not the role or function of BBC news to ‘support’ any argument but rather to provide a platform for a range of views to be heard.

This is a hugely important point, much researched in academia across Europe and the US and central to any explanation which goes beyond my account to identify causes. Journalists are known to be subject to peer pressure. Junior reporters work toward, perhaps sub-consciously, the approval of their seniors.

Members of media elites (owners, editors, heads and directors) interlock socially with other elite members (politicians, officers, judges, directors, CEs) via selective education, early careers  and social/cultural membership. No conspiracy is required. Elite members act in their own interests and those interests are the same as the interests of their groups. Thus thought control in liberal democracies is made possible and far more subtle than in totalitarian states where the people know, always, not to trust their media.

You misunderstand yourself and the BBC. Impartiality is not attainable. Like many academics, I practice self-awareness but even then I make no claims.

The report does offer a definition of ‘statements’.  It also notes that, in news stories, they “were rarely more than one sentence in length with the presenter, interviewer and multiple public figures generally constrained to enable inclusion of all parties (our emphasis).  For the BBC, the composition of broadcast reports is based on editorial rather than numerical criteria, with access afforded to a range of voices and opinions.

OK, good

Abusive of Pro/Abusive of Anti
Broadcasting the use of insulting language aimed at pro-independence/anti-independence campaigners

No definition of ‘insulting’ is offered, nor does the report indicate if the terms ‘abusive’ and ‘insulting’ are interchangeable, nor if either or both are to be regarded as one and the same as ‘offensive’, to which the report refers in Question 7.

Again, absence of clear definition does not allow the reader to understand what is meant by such in this report.

They seem the same to me.

Section 2


(a) Use of statistics

The findings of the report are initially presented in the form of statistical data within a table. With no other information provided to allow informed decoding of these statistics, it is impossible to understand what the figures actually represent.

No other information? What about the definitions of the coding categories on the previous pages?

That said, the report claims the table presents data that “can be used to reveal the distribution, over 12 months, of different types of message within broadcasts..”  It contains no information about distribution – what it provides is a numerical count, relative to an overall timeframe: there is no indication of the spread of stories, per day, week or month, to which these coded categories apply. The only indication it offers of the distribution of these ‘messages’ is to note that news reports relating to the referendum were either “fairly regular occurrences” (BBC Scotland/STV) or “rarely reported” (BBC/ITV). 

The lack of precision in definition and use of language again does not assist in interpreting the data.

This refers to the distribution of the statements across the coding categories in a study over 12 months. With some effort I can misread the above your way.

Following the contention that referendum stories were “rarely reported” on network news, the report concludes that this reveals “apparent disinterest in a major constitutional challenge to the very existence of the UK, by its two dominant news programmes…”

The report arrives at this conclusion on the basis of a simple mathematical calculation and, in so doing, ignores any other possible factors at work.

The appearance of such stories on network news may, for example be influenced by the time period over which the research was carried out (the research covers September 2012 – September 2013 – the date of the referendum was not announced until 21 March 2013, six months after the research project began). Another factor could simply be that such stories were considered less relevant to those ineligible to vote (in other areas of the UK) at a time when the referendum vote was one to two years away.

The influence of such factors is ignored in this research in favour of a simple numerical interpretation of data, analysed in crude quantitative terms. The conclusions, such as that noted above, largely go beyond what the available data confirms.

All of the above are possible, I agree, but they are all, also, further evidence of disinterest in a forthcoming event which poses an existential threat to the UK.

The focus on numerical interpretation is restated in the report’s next contention, that “The simple numerical preponderance of anti-independence statements over pro-independence statements by a ratio of c3:2 on Reporting Scotland and on STV, is also clear.”

In this instance, the report offers a possible explanation of this, as “the editorial decision to allow all three anti-independence parties to respond to each SNP statement creating an unavoidable predominance of statements from the former..”.

We would make two related observations. First, we would not accept that, by allowing the other parties to comment on referendum issues, this, in any way, is evidence of bias or lack of ‘fairness’. Most would regard such an inclusive approach as evidence of steps taken to allow the range of voices to be heard. 

Agreed; I offered the explanation in the same spirit and do not tie this behaviour to any suggestion of bias or lack of fairness.

Secondly, it is simply not correct to suggest that “all three anti-independence parties (are allowed) to respond to each SNP statement (our emphasis)”.

If that contention were correct, taking the report’s own ‘evidence’, the table noted above would show that, for the 171 pro-independence /SNP statements the report claims appeared on Reporting Scotland, the number of anti-independence/SNP statements would be 513 (171 x 3), not 262.

By ‘each’ I do not mean ‘every’. This is getting desperate!

Political comment on such issues is sought according to the requirements of the story, with balance achieved over a period of time rather than necessarily within any one programme or news item. Given the constraints of time and other factors which impact on the construction of every broadcaster’s news bulletins, this is normal practice across the industry.

What steps do you take to ensure balance over time? Why is there, then, not balance over the 12 months reported?

(b) Anti-independence and economic affairs: examples cited

The report then claims that “Anti-independence statements were heavily concentrated on economic affairs…” and cites various examples to lend credence to its contentions. 

To take these in turn:

Example 1

The report refers to a Reporting Scotland item on Trident on 29/10/2012 which it claims “is driven by a weight of one-sided and unchallenged evidence and commentary”, cites “unnamed economic advisers” who are “allowed to suggest 6500 jobs (could be) lost if Trident goes and an overall cost of £20bn while the report finishes ominously with ‘Whitehall could play hardball’.”

The Trident item consists of a report by BBC Scotland political correspondent Niall O’Gallagher, related to the visit of Secretary of State for Defence Philip Hammond MP to the Clyde base, followed by a studio interview with BBC Scotland Business and Economics Editor, Douglas Fraser.  Both reports accord fully with the requirements of our Editorial Guidelines. 

The jobs figure is cited, in person, by Douglas Fraser, not by some “unnamed economic advisers”.  In the broadcast report, he explains, using graphics, that, under current plans, that figure could go up to 8000, then says: “…but, you could look at it another way, as Nicola Sturgeon was doing; if you take the same money and spend it on other priorities, whether it’s military or civilian, you could create at least as many jobs.”

The £20bn quoted is not the cost to the economy if Trident goes, but rather the cost of replacing Trident (which Douglas Fraser attributes to the MoD, pointing out that it’s a six-year-old figure and the MoD’s record on remaining within budget is not good.  He also mentions the operating costs of £1.5bn pa).

The report author then notes that the item “…finishes ominously with ‘Whitehall could play hardball’”

Aside from the questionable use of such value-laden terminology as ‘ominously’, this is simply incorrect.  It doesn’t.  It finishes with Douglas Fraser quoting Alex Salmond’s suggestion that a new home could be found for Trident in the USA or France.

I do not question the presence of opposing views here but the weight in this piece was tilted toward uncritical presentation of the MoD arguments.

What then is the source of the figures reported by Douglas Fraser if not ‘unnamed economic advisers’?

I find the notion that Whitehall might play hardball, quite ominous and I think it was meant to sound that way too.

The Alex Salmond comment does not represent the finishing statement in a sequence of ideas but rather is a throwaway piece with humorous intent. I laughed.

Example 2

Energy prices
The report refers to a Reporting Scotland item on 11/12/2012. It claims: “the programme opens with ‘Row over independence could lead to higher electricity bills’ then runs through a series of negative sound bites interspersed with SNP protest – ‘questions mount over independence’, ‘UK government claims cost could rise’, ‘Could Scots customers have to pay more?’, ‘Labour spokesman – danger’ before allowing the evidence of Scottish over-production, renewables and a captive market in England to cast serious doubt on the motivation for the initial headline ‘scare’.

There are a number of significant flaws in this summary. 

1. The opening: the programme actually opens with the headline ‘As the row over independence continues to brew, claims that it could lead to higher electricity bills.’

2. The report then claims the programme “runs through a series of negative sound bites”.   The programme presenter then runs through the rest of the headlines, before returning to the main story.

3. In respect of the “negative sound bites” claimed, the report offers the following examples: 

* “questions mount over independence” – this line is not used in the broadcast, either in the headlines at the beginning or during the report;

* “UK government claims cost could rise”, “Could Scots customers have to pay more after independence?”- these are not sound bites – they are part of correspondent David Miller’s script, describing the parameters of the debate;

* “Labour spokesman – danger” – this is simply wrong.  No Labour spokesperson appears in or contributes to the broadcast report.  

Liberal Democrat MP and Secretary of State for Energy and Climate Change Ed Davey MP makes a remark about danger.

4.  “…before allowing the evidence of Scottish over-production, renewables and a captive market in England.” 

The entire broadcast report is about 1′.50” in length– of that, the first 45 seconds describe the context of the debate, the UK government claims and it includes a clip of Ed Davey MP explaining those claims. In the remainder of the broadcast report, there is a description of the Scottish government’s counterclaim and then a clip of Scottish Government Energy Minister Fergus Ewing MSP elaborating on this.  The broadcast report ends with David Miller citing industry sources, saying that the effect of independence on electricity bills is hard to predict but that those sources add that “it’s inconceivable that England would stop importing renewable energy from Scotland. Energy policy often sparks controversy – today it’s also generated a political row.”

The overall presentation remains, in my mind and I think for any reader of the above, weighted to emphasise risks for which no evidence is provided. Why was this story chosen and presented in such dramatic terms. It could have been ignored (evidence-free as it is) or presented in a less dramatic way.

To suggest that an editorial decision was taken to structure the piece in such a way as to concentrate on ‘anti-independence’ aspects of the story before allowing (our emphasis)…evidence…to cast serious doubt on the motivation for the initial headline ‘scare’” is not borne out – in any way – in analysis of the broadcast package.

The example cited therefore in no sense supports the contention of the author.

I do not suggest, anywhere, the presence of editorial conspiracies. As explained earlier, journalists work semi or unconsciously in response to peer pressure and to satisfy perceived impressions of the expectations of superiors. I feel confident that yes-sympathetic viewers will have been unnerved by this broadcast and that no-sympathetic viewers will be further embedded in their perceptions.

Example 3

Cancer treatment
The report then turns to broadcast reports on health issues.  It says: For example, on 27/9/12 the case of a Scottish patient seeking free cancer drug treatment only available in England was highlighted and linked to the relative lack of GP control in Scotland.

First, this story was not related to the referendum, nor at any point in the broadcast piece was this suggested. It was about the respective changes in the NHS in England and Scotland and the effect on patients.

Of course it was not explicitly related to the referendum by the reporter but, by making unfavourable comparisons between the Scottish and English NHS, it unavoidably contributes to the debate. The decision to choose a one-off case and not to balance with stories of English patients feeling pressure to move to Scotland for treatment, or other balancing ideas, leads to bias.

Secondly, the report is categorically not about free cancer drug treatment. Cancer is not mentioned once – the patient case study, Beth Butterfield, has lost confidence in her local doctors and wishes to be sent elsewhere. The problem she has is with the difficulty of transferring hospitals, which in theory would be easier in England due to the new reforms. What Ms Butterfield is suffering from is never explained.

“This began a mini-series of reports on alleged failings in the Scottish NHS by Reporting Scotland reporters and by Labour spokespersons.”

This refers to a one day series of reports (‘NHS Day’– 27/9/12) on BBC News, broadcast across the UK over 24 hours. It was not about “alleged failings in the Scottish NHS”.

This story read to me as a quite unbalanced take on the relative developments in the Scottish and English NHS. It was as you note not specifically about cancer care but care generally. You write ‘categorically not about free cancer drug treatment. Cancer is not mentioned once’  Another of your substantive errors, I take it?

I read it as drawing attention to failings in the Scottish NHS. As I reply, a third-party has drawn my attention to another independent study highlighting ‘censorship and distortion’ in BBC coverage of NHS reforms in England (Huitson, O. (2012) ‘How the BBC betrayed the NHS: an exclusive report on two years of censorship and distortion’  http://www.opendemocracy.net/ourbeeb/oliver-huitson/how-bbc-betrayed-nhs-exclusive-report-on-two-years-of-censorship-and-distorti).

The above research concludes:
“The BBC routinely described the Bill as a reform to empower GPs – the government’s description – rather than as turning the NHS into a market driven by shareholder interests, which was what the critics maintained – accurately, as is now becoming clear.  The BBC’s public service remit should surely have required it at least to present the Bill’s purpose as contested.”

The same criticism can be reasonably applied to the Reporting Scotland story. Why was this story developed so soon after the publication of the Huitson paper?

(c) Use of expert advice and ‘evidence’

The report then makes a wide-sweeping claim that it found “little evidence” of the use of “independent, academic or scientific” sources in referendum reporting “and where there was, there was clear tendency to use anti-independence over pro-independence evidence.”  Expert sources are regularly used by BBC Scotland where and when it is relevant and appropriate to do so and/or they are cited within reports offered by BBC correspondents and editors.

Reporting Scotland referred to such sources only 26 times out 365 broadcasts. Of these 22 offered evidence against the yes position.

To suggest that the “tendency” is to use “anti-independence over pro-independence evidence” is another un-contextualised claim made without reference to supporting evidence.

22 references to reports from The Treasury,  IFS, OBR, MoD and only 4 from the Scottish Government suggests a very marked imbalance.

The proposals for independence, as laid out in Scotland’s Future, published by the Scottish Government, are currently setting the terms for the debate and are consequently receiving a considerable degree of scrutiny.  Such proposals as the opponents of independence bring forward, either collectively or individually, will also be subject to debate and analysis.

‘Will be’ is not good enough. Why have they been presented so tamely so far?

These proposals for change, from either side, will inevitably attract a greater degree of scrutiny than the status quo. 


However, to label such interrogation of evidence as ‘anti-independence’ is to fail to take account of the context within which such activity takes place.

To support its contention that “there was clear tendency to use anti-independence over pro-independence evidence” the report cites the following example: “a Glasgow University academic was ‘outed’ as having been ‘bought’ by the SNP to support the independence case (Reporting Scotland, 21/8/13).”

We would make three points here regarding accuracy:

* the date is wrong – the piece in question was broadcast on 22/8/2013, not 21/8/2103;

OMG as the young people say.

* the word ‘outed’ appears nowhere in the broadcast;

A pretty good choice of vocabulary I think to describe the report but not in the report, nor quoted in an attempt to deceive the reader.

* and it was the Yes Campaign (as stated in the programme), not the SNP (as stated in the UWS report), who paid Dr Bulmer. 

Again, is this all you’ve got?

Dr Bulmer appeared in the programme and robustly defended his position. We would regard the broadcast report as entirely accurate and impartial. In both the broadcast report and the commentary that followed, equal attention was paid to the allegations that the Yes Scotland campaign communications were hacked – this was not just a story about Dr Bulmer’s fee.

My full comment was: ‘Indeed the IFS was referred to as a ‘well-respected think tank’ (Reporting Scotland, 19/11/12) whereas a Glasgow University academic was ‘outed’ as having been ‘bought’ by the SNP to support the independence case (Reporting Scotland, 21/8/13). I stand by my interpretation of the ‘outing’ but the main point was about balance in the respect accorded to different sources.

The UWS report then looks at “The sequence of statements whereby anti-independence arguments preceded pro-independence responses as opposed to the reverse order” of which it says “there was a clear majority of the former” on Reporting Scotland.  Again, without the data available either within the body of the report or appended to it, it is difficult to assess the merit of such a contention.  Likewise, without knowledge of what was coded as an ‘anti-independence’ statement or story (see, for example the earlier comments on inappropriate labelling relative to the NHS story, on P9) it is not possible to determine what validity this contention may or may not have.

The same would be true of a report lauding the impartiality of the BBC. Maybe you’d trust the author? However, a list published elsewhere suggests a wider tendency to launch stories with a ‘warning’ to potential Yes voters (Murray, C. (2012) ‘BBC the New Hammer of the Scots’. http://www.craigmurray.org.uk/archives/2013/04/bbc-the-new-hammer-of-the-scots/)including:

‘By contrast, there are no BBC headlines in this period that promote positive claims about Scottish Independence. You will look in vain for headlines that say “Yes campaign says independent Scotland will be eighth richest country in the world” or “Official GERS report shows Scotland’s public finances much healthier than those of the UK”. Such headlines just do not exist. Reporting Scotland or Newsnight Scotland has never, never been led by a positive story about independence. It has been led on dozens of occasions by the negative.’

The report then contends that “The Reporting Scotland imbalance tends to normalise the No/anti-independence position and put the onus on the Yes/pro-independence position to justify itself.”

The use of the word ‘normalise’ is interesting – Scotland is currently within the UK; the Scottish Government is seeking to change that position and has put forward a series of proposals to the electorate; these proposals are, quite rightly, under a good deal of scrutiny, with the Yes campaign and others seeking to elaborate on what would be different from the current ‘norm’. 

This is a conservative reading which might reveal ideological positioning. Scotland’s position within the UK is no more normal than it would be outside. The dominant trend (norm?) in the last hundred years has been for imperial breakup and the emergence of new states. It is only correct  to impose higher standards of scrutiny on proposals for change if you think the current situation is demonstrably superior.  In the interests of fairness, why not expect the No campaign to make the case for the Union in response to critiques of the British state from those who favour independence?

To infer that somehow Reporting Scotland, in its construction and editorial decision-making, is supporting one side over the other and in so doing, influencing the electorate in one direction relative to the independence debate, is yet another suggestion without substance. 

As repeated earlier, I’m only suggesting that it’s happening. I know you don’t get round the table and plot the downfall of the Yes campaign. You don’t do you?

(e)  Personalisation, demonisation and the undermining of individuals

The next part of the UWS report equates, in a rather simplistic way, ‘personalisation’ and ‘demonisation’ which it suggests, in broadcast reports, has the effect of ‘undermining’ the individual: it then applies this theory in its interpretation of Reporting Scotland output. 

In this process, it notes that “The tendency by opposition politicians to attempt to undermine the Yes campaign by labelling its ambitions as Alex Salmond’s desires is, in part, beyond the editorial role”. Nevertheless it argues that the broadcasters are complicit in this process – “it was common for reporters and presenters to adopt the same style:”

Independence is the stated aim of the Scottish Government, of which Alex Salmond, as First Minister, is the head. He is, as such, a recognised voice for the independence argument and he has publicly stated his personal and political belief in independence.

No one such figure is as easily identifiable for the anti-independence argument, given that three main political parties are involved, each with its own political agenda.

However, to suggest, as the report does, that for the BBC, as broadcaster, to present Alex Salmond’s personal and political desires as one within the independence argument and that that is, somehow, part of a strategy to “undermine” him as a political figure, is yet another argument wholly without substance.

I think it is a strategy by the other parties and their strategic advisers to weaken the yes campaign by this technique. I think the historical examples I give show that it works. I don’t think the BBC reporters are likely to be part of this plan but rather that they have lazily adopted it. It’s not good enough to say no one figure is as easily identifiable for the No campaign. It would be simple to stop saying Alex Salmond so much and just refer instead to the Scottish Government or the Yes Campaign.

In this respect the report also points to the broadcasting of ‘insults’, the preponderance of which it claims were aimed at Alex Salmond.  However no substantive examples are cited, other than a reference to “personally insulting comments by anti-independence representatives (especially Johann Lamont) aimed at Alex Salmond.”

Virtually all of these were from the other three party-leaders. Why not edit them out thus giving more time for substantive points?

Leaving aside the concerns, noted earlier, about what constitutes ‘insulting language’ and the undefined criteria used to identify such language, we would note that robust discussion and strongly voiced opinions often form part of First Minister’s Questions and Prime Minister’s Questions.  What we broadcast from the Chamber is determined on editorial grounds.  In this, as in all of our broadcasting, we accord fully with the BBC’s Editorial Guidelines and the Ofcom Broadcasting Code.

That, as the report notes, “the use of insults aimed at Salmond declined and had become less common in the second six months of the survey” is perhaps the result of the fact that fewer “insults” (however the report has chosen to identify these) were being made, rather than the decision of the BBC not to report them.

(f)  The approach of reporters and presenters

To support the theory that reporters and presenters had adopted the “same style” as “opposition politicians to attempt to undermine the Yes campaign by labelling its ambitions as Alex Salmond’s desires”, the report cites a number of examples.

To take these in turn:

Example 1

On 23/10/12, in Reporting Scotland, ‘Alex Salmond under pressure!’

The full headline was ‘Tonight, Alex Salmond’s under pressure over independence.’

We would be interested to understand how the exclamation mark, above, was translated from the actual broadcast piece. This suggests a subjective interpretation of a broadcast headline, which was then transcribed as thus – not what one would expect to find in what should be an ‘objective’ analysis of news output.

The background to this story was that Alex Salmond was personally under pressure over allegations he had lied – this was not an instance of equating independence policy with Alex Salmond.

The text of the broadcast piece is as below: 

Jackie Bird: “Welcome to Reporting Scotland. Tonight, Alex Salmond’s under pressure over independence.

Clip from Andrew Neil/Alex Salmond interview.

Jackie Bird: The First Minister denies opposition claims he lied in that BBC interview about legal advice over EU membership.

Alex Salmond (in Chamber): I hope at a suitable opportunity those, admittedly outside this chamber, who have made these assertions will have the courtesy and integrity to withdraw them.

We fail to understand how this serves as an example of the complicity of reporters/ presenters in demonising or undermining an individual.

This was used with others to highlight the repeating tendency, especially in headlines, to refer to Alex Salmond by name when it could have been First Minister or Leader of the SNP with the latter more meaningful associations with his role rather than his persona.

Example 2

On 23/10/12, in Reporting Scotland, Willie Rennie (Lib Dem) ‘challenged Alex Salmond’s policy’.

This quotation does not appear on Reporting Scotland on this day.

Willie Rennie did not appear in the 1830 Reporting Scotland, though Ruth Davidson did.

OK, my mistake.

Example 3

On 25/10/12, in Reporting Scotland, Salmond is described by Johann Lamont (Labour) as ‘straight as a corkscrew’ and then compared by Willie Rennie (Lib Dem) to bent salesman ‘Delboy’.

This is a report by Brian Taylor on that day’s First Minister’s Questions, where the main subject under discussion was a forthcoming inquiry into Mr Salmond’s conduct in relation to legal advice on future membership of the EU – an inquiry set up by the First Minister himself. 

As in the example cited above, Willie Rennie did not appear in the 1830 Reporting Scotland, though Ruth Davidson did.

It is Ruth Davidson, not Willie Rennie, who compares the First Minister to ‘Delboy’.

The comment referred to, and Johann Lamont’s remarks, were delivered in formal session in the Chamber of the Scottish Parliament. 

The two remarks are followed by a clip of Alex Salmond (also from First Minister’s Questions) responding to these remarks and explaining how in his opinion the misunderstanding arose.

The end of the report is as follows: “Mr Salmond is adamant that he gained the trust of the Scottish people at the ballot box and will sustain it through to the Referendum.”

In short, not one of the examples cited substantiates the claims made within the report.

Unimportant. Reporting Scotland chose to edit in offensive statements at the expense of more important matters. This has the effect of reinforcing processes of demonisation.

(g)  The role of political editors in framing the debate

The report then moves on to discuss the “distillation of the debate over independence into a largely economic debate” and says that “Particularly notable is the role, here, of political editors in framing the debate in this way, telling the viewer that the debate over living standards, employment and taxation was the only debate anyone cared about. No evidence for this view was given.”

The report provides no evidence to support this contention.

Brian Taylor repeatedly does so.

Social attitude studies have made it clear for some time that the economic debate is the one that voters deem most important and is one aspect of the debate that is of significant interest to them. 

Not mentioned in reports.

Note, for example, the latest ScotCen Central Research Study (January 2014) which highlights this very point. [3]Tiny sample size, problematic validity,

Political editors did not decide to “frame” the debate in this way and to suggest they did is wholly misleading. No evidence was given.

(f)  The impact of ‘closing statements’

Attention is then turned to “closing statements in reports” which it suggests (with no evidence to support such a contention) might be felt (our emphasis) to leave a lingering impression and thus carry more weight than some others”.  Leaving aside the questionable validity of this contention, qualified as it is by a vague suggestion made without reference to any evidence which would substantiate it, the report then suggests that the preponderance of closing statements that are anti-independence are to be found on Reporting Scotland. In support of this argument it cites a number of examples.

Example 1

On 27/9/12, in Reporting Scotland, a piece on the changes to the NHS in England was used to suggest that the Scottish system’s reluctance to change ‘is bad news for Scotland’ and finishes with the unsubstantiated suggestion that GPs and patients might be ‘planning to move to England’(our emphasis).

As noted previously (Page 9), the example noted above is not about the referendum. It is about the respective changes in the NHS in England and Scotland and how those changes affect patients. There is no mention or suggestion at any point in the broadcast piece that this is linked to the referendum.

The correct version of the citation referred to within the UWS report is actually: “The changes in England have attracted a lot of opposition, but some think Scotland’s reluctance to change is bad news for patients.” It is, thus, not stated as fact; it is a claim which is examined in the piece.

The report deals with reforms giving English GPs more power, but also mentions that a recent Scottish reform gives Scottish patients more power by sending the money for their treatment directly to their bank accounts.

The report ends with the sentence: “Despite this, Beth is now planning to leave her Dumfriesshire home, and move to England. She hopes the system there will give her better care.” This refers only to Ms Butterfield, the interviewee, who is planning to move.  Contrary to the UWS report contentions, it makes no claim about other patients or GPs.

Topics such as this are unavoidably linked to the referendum. The decision to choose to allocate time to this story of one person who turned out to be a Labour Party plant, at the expense of the several cases presenting the Scottish NHS in a better light, in English newspapers looks biased.

Example 2

On 5/10/12, in Reporting Scotland, the Scottish Government’s commitment to universal benefits was immediately followed by a reference to ‘spending watchdog chief Robert Black who has questioned whether such benefits are affordable’ and reinforced by reference to Black’s cv –’few people are better placed to understand the challenges’.

This report is not about the referendum. There is no mention or suggestion at any point in the broadcast piece that this is linked to the referendum.  For accuracy, it should be noted that the reference in the broadcast report was to “former spending watchdog Chief Robert Black…”

The text of the broadcast piece is as below: 

Sally Magnusson: The Scottish government has reaffirmed its commitment to benefits like free prescriptions and free personal care for the elderly. It was responding to the former spending watchdog chief Robert Black, who’s questioned whether such policies are still affordable. Our political correspondent Raymond Buchanan joins me now. Raymond. “

(The full reference to (Mr) “Black’s CV” immediately follows within the piece):

Raymond Buchanan: Sally, few people are better placed to understand the challenges public finances face than Robert Black. As Auditor General, his job was to study the statistics, and some trends jump out from the page…”

These are the opening lines of the report.  They certainly do not, as the reports claims, constitute a ‘closing statement.’

The actual closing statement within this report is:

Raymond Buchanan: But the challenges aren’t confined to an ageing population, Robert Black says there’s a four billion pound backlog for maintenance in the public sector, another sign that as budgets go down, costs are still going up.” 

As neither the referendum nor independence has at any point been mentioned in this piece, this last line can hardly be considered ‘anti-independence.’ 

If as Brian Taylor tells us, the referendum is all about the economy then Lord Black’s intervention is obviously meant to influence the debate on independence. The very respectful treatment of Lord Black’s views suggests at least passivity in BBC staff in the face of elite representatives. The choice of Black’s report and not the more optimistic paper from the Jimmy Reid foundation on the same topic is, perhaps, revealing.

Example 3

On 26/4/13, in Reporting Scotland, a generally negative assessment of the future of insurance companies after independence finished with the Labour spokesperson’s assertion of ‘billions in costs’ and ‘potential closures’.

This broadcast piece is about pensions, not insurance companies.

The reference to closure is surely about insurance company offices.

The piece firstly sets out the context of pension schemes’ deficits (vital to the understanding of all that follows).   The piece then describes the challenges set out in the report, with an interview from David Wood from ICAS, the Institute of Chartered Accountants of Scotland.  We then hear from John Swinney, followed by Gregg McClymont of Labour: “That means billions of extra cost for the companies involved, or it means if they don’t want to meet those costs or can’t meet those costs, it means potentially closing the schemes in Scotland.” 

These words from Gregg McClymont MP do close the broadcast report – 

That’s what matters.

– however, the studio discussion on this story continues after that report.

David Henderson’s last line is actually about ICAS:  “And true to form, they’re urging the governments in Edinburgh and London to do some careful pension planning before next year’s referendum. Sally.” 

Attempting to soften a blow after it’s been delivered will be of little effect. Again, it’s the repeated pattern of shock news for Yes supporters followed by defensive responses which are disadvantaged by the sequence.

Once again, the examples cited do not support the contentions made by this report.

Section 3

Concluding Remarks

The report closes on a distillation of many of its earlier arguments, though, again with no evidence cited, it makes further allegations, that Alex Salmond is “often portrayed as selfish and undemocratic” and concludes that Reporting Scotland alerts “typically” contain “a Westminster scare story, on the Yes campaign” that are “mostly left unanswered and unchallenged”.

To suggest this is to suggest that BBC Scotland is constantly in breach of the BBC’s Editorial Guidelines (and Ofcom’s Broadcasting Code), its news decisions determined by an editorial imperative to favour one argument over another, the result of which would be hundreds of editorial transgressions every year.

It is a suggestion completely without merit.

The UWS report’s closing remarks are worthy of consideration. It refers to its conclusions as based “on the objective evidence presented here”. 

There is little, if any, objective evidence presented within the report: on the contrary, it is, in fact, highly subjective and selective in its approach and in its assessment of broadcast output.

As discussed earlier I accept a degree of subjectivity is unavoidable despite my best efforts. My evidence is as objective as that produced by any research in the field of social and political research. I suspect that it compares well with the pronouncements of those charged to defend corporate interests.

Finally, for it to suggest that the “evidence” it claims to have uncovered points to a conclusion that “coverage seems likely to have damaged the Yes campaign” is to draw the report to a close on another wholly unsubstantiated contention, which is based on conjecture rather than any empirical evidence.

There is no evidence whatsoever in the report that it has conducted any research or that it has otherwise collected substantive evidence that bears out this conclusion. 

It does not indicate what it means by ‘damage’ – reputational, in terms of voting intentions, etc?   If the latter, how can it arrive at that conclusion without having first assessed the voting intentions of the electorate and then clearly demonstrated a causal link between changes in those intentions, over a period of time, and the direct influence of broadcast output over that same period? 

In short, the report concludes with a guess. 

Reporting Scotland’s Brian Taylor is regularly asked what the impact of events such as the recent Bank of England governor’s speech in Scotland will have. He rarely hesitates, he gives only anecdotal evidence and is left unchallenged before a mass audience. No harm to him, but I think my evidence base is somewhat stronger.

Overall, it fails to provide any evidence, at all, that would support its main contentions.

Dr John Robertson says:

I think I’ve answered all the questions needed to contest these conclusions. Social research is unavoidably subjective to some degree but we have done our best to reduce the level. The BBC response is a remarkably heavy-handed reaction. Why did they not report the research, let their experts critique it on air and then ask me to defend it? Instead we see a bullying email to my employer and a blanket suppression across the mainstream media in the UK. I’m shocked.

There have been in excess of 30,000 hits on internet reports linking to my research. I’ve had more than 100 direct emails and around 400 blog responses suggesting a wide popular discontent with TV coverage of the Scottish Independence Referendum and in most cases suggesting I’ve understated the imbalances and bias. I’ve had only one comment really disagreeing with me.

I started out to gather the data, code it, sort it and report it in the professional and impartial style I have learned in 30 years of higher education. I deliberately avoided reflecting too much as I worked. Remember, I spent most of my days on unrelated tasks. The level of imbalance and the presence of propagandising techniques such as the demonisation of Alex Salmond, which emerged as I processed the first year’s data, did not surprise me. Remember I am a student of propaganda and media compliance in more oppressive contexts such as Russia and the Middle East or Europe in the early 20th Century.

My team is small, low budget and part-time. I take responsibility for a small number of factual errors in writing up but these are of no consequence for the overall conclusion which is that Reporting Scotland and STV have produced over one year an account of the independence debate which favours the No campaign.

Appendix 1

Relevant extracts from BBC Editorial Guidelines

Section 4: Impartiality

Impartiality lies at the heart of public service and is the core of the BBC’s commitment to its audiences.  It applies to all our output and services – television, radio, online, and in our international services and commercial magazines.  We must be inclusive, considering the broad perspective and ensuring the existence of a range of views is appropriately reflected.

The Agreement accompanying the BBC Charter requires us to do all we can to ensure controversial subjects are treated with due impartiality in our news and other output dealing with matters of public policy or political or industrial controversy.  But we go further than that, applying due impartiality to all subjects.  However, its requirements will vary.

The term ‘due’ means that the impartiality must be adequate and appropriate to the output, taking account of the subject and nature of the content, the likely audience expectation and any signposting that may influence that expectation.

Due impartiality is often more than a simple matter of ‘balance’ between opposing viewpoints.  Equally, it does not require absolute neutrality on every issue or detachment from fundamental democratic principles.


4.2.1 We must do all we can to ensure that ‘controversial subjects’ are treated with due impartiality in all our output.

4.2.2 News in whatever form must be treated with due impartiality, giving due weight to events, opinion and main strands of argument.

4.2.3 We seek to provide a broad range of subject matter and perspectives over an appropriate timeframe across our output as a whole.

4.2.4 We are committed to reflecting a wide range of opinion across our output as a whole and over an appropriate timeframe so that no significant strand of thought is knowingly unreflected or under-represented.

[1] http://www.bbc.co.uk/editorialguidelines/

[2] http://stakeholders.ofcom.org.uk/broadcasting/broadcast-codes/broadcast-code/fairness/