Reporting Scotland: j’accuse – accuracy and fairness on maths figures

24
5046

Media analyst John Robertson takes issue with the BBC’s coverage of a major education story

‘Critics describe a steady decline in pupil attainment in maths and arithmetic as ‘appalling!’ (Reporting Scotland, 31st May 2016)

This headline, on the Scottish Survey of Literacy and Numeracy 2015 (Numeracy), is simply wrong. The Scottish Survey of Literacy and Numeracy is a Scottish Government initiative started in 2011/12. There is no statistically valuable evidence of educational change to be drawn from data collected every two years, with very low single-digit changes between collections, and covering such a short total time as four years. Try submitting an undergraduate university dissertation proposal based on this and watch the supervisor laugh till he hurts.

Dr John Robertson
Dr John Robertson

I’m the one accusing BBC Scotland of decline and I have the credentials to do so, having published in the top, peer-reviewed, media-analysis journals. The ‘critics’ accusing the Scottish Government’s performance on educational attainment are the usual suspects, close to the BBC’s heart, members of the Tory and Labour parties. These politicians have little credibility having never published in peer-reviewed educational research journals and, in most cases, never taught a class of 30 wean-bairns. I’ve done both. The teachers’ union, the EIS, offered a more intelligent analysis, linking these figures to wider austerity, but only appeared on STV. No serious academic researcher was daft enough to join in the feeding frenzy. Actually, it was no academic researcher at all.

Here are the actual figures from the full report, available online, in graph form:

Numeracy

The above graphic is from Chapter 3: Numeracy attainment over time at: http://www.gov.scot/Publications/2016/05/2836/4

Here is how this graphic is interpreted on the gov.scot site and indeed should be interpreted:

The proportion of P4 pupils performing well or very well decreased between 2011 and 2013, and again slightly between 2013 and 2015.

The proportion of P7 pupils performing well or very well decreased between 2011 and 2013 but remained constant between 2013 and 2015.

The proportion of S2 pupils performing well or very well remained stable between 2011, 2013 and 2015.

I’m not suggesting this is good. It is evidence of some ongoing decline but also of a deceleration of that decline or even of its flattening out. The figures from deprived areas are worrying. However, they are not as described, ‘appalling’.

Stage 2011 to 2013 2013 to 2015 2011 to 2015

P4 Lower in 2013 Lower in 2015 Lower in 2015

P7 Lower in 2013 No difference Lower in 2015

S2 No difference No difference No difference

Table 3.3 ‘Most deprived category’ (pupils from the most deprived 30% of datazones)

The change from 2013 to 2015 (middle column) is important because it may reflect more recent changes in policy which we need to be able to evaluate. These suggest a flattening out of a decline. Now if that decline is one which goes back well before 2013, it’s stalling, decelerating or flattening might be a sign for optimism of the type not familiar to BBC Scotland if they think it might reflect well on the SNP. The good news is that we can go back much further, to 2000 AD, deep into the long days of New Labour Scotland Branch rule and, presumably, better attainment? These data from PISA (Programme for International Student Assessment) cover a broader range of mathematics skills. See this from PISA 2009 which reveals a decline going back at least to 2000.

Year Mean Score

2000 533

2003 524

2006 506

2009 499

These figures can be found in: http://www.gov.scot/Publications/2010/12/10141122/16

I’ll update on PISA 2012 below.

So, what we have here is evidence of a longer decline perhaps caused by New Labour policies and being slowed, if not stopped, by the Scottish Government’s actions in the last few years. Remember, changing educational outcomes takes time, decades even.

You’ll remember, only last week we heard, erroneously, that disadvantaged young people in Scotland were actually less likely to enter higher education than disadvantaged young people in England. See my NN piece last week for more on this. BBC Scotland does like a comparison between our education systems or our health services. Let’s see if PISA had any such comparisons to help us all contextualise and to update from PISA 2009, the Scottish data.

‘Scotland was the best for reading and maths of the UK nations, ranked 15th and 21st (out of 67 developed countries)

That’s a BBC headline at: http://www.bbc.co.uk/news/uk-wales-11930257

Here’s a bit more from the actual PISA 2012 report with page numbers in brackets:

Scotland had the smallest percentage of pupils working at the lowest levels in all three subjects and their low achievers scored more highly in all subjects. (83)

In all subjects, Scotland had the lowest percentage of pupils at Level 1 or below (83)

The highest attainment for mathematics was in Scotland (84)

Surely, just a wee smile is justified here? Have some more:

The distribution of performance in mathematics showed that there was some degree of variation around the mean score for mathematics in all countries, as would be expected. The size of this variation indicates the extent of the gap between low and high attaining pupils. The OECD average score was 301 points. The range was wider than this in England and Northern Ireland and narrower in Scotland and Wales. The highest difference of 316 was found in England. (87)

So, in the UK context, Scotland has a narrower attainment gap than England. Further, clear evidence of this is:

Scotland had the lowest percentage of pupils working below Level 1 in mathematics (4.9 per cent). This compares with the OECD average of 8.0 per cent. In England and Northern Ireland the proportion of pupils working at the lowest level of proficiency in mathematics was close to, or the same as, the OECD average (8 and 8.6 per cent respectively). (88)

Scotland had the narrowest range of attainment and the scores of their lowest achieving pupils were much higher than those in the rest of the UK or the OECD on average. (99)

The above can be found in more detail at:

https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/299658/programme-for-international-student-assessment-pisa-2012-national-report-for-england.pdf

‘They’ve had enough’, you’ll be saying but remember this is the England we so often find BBC Scotland fawning over and throwing at us deluded independence supporters, so here are more examples:

Young people in England are the most illiterate in the developed world with many students graduating with only a basic grasp of English and maths, an in-depth analysis by the Organisation for Economic Co-operation and Development has found.

The OECD report rated English teenagers aged 16 to 19 the worst of 23 developed nations in literacy and 22nd of 23 in numeracy..

http://www.ibtimes.co.uk/young-people-england-have-lowest-literacy-levels-developed-world-says-oecd-1540711

Finally, returning to the PISA reports, to further put the Scottish trend (pulling out of a long decline?) in context:

‘Of the 64 countries and economies with trend data between 2003 and 2012, 25 [only] improved in mathematics performance.’ (4)

https://www.oecd.org/pisa/keyfindings/pisa-2012-results-overview.pdf

So, having looked more thoroughly, over a reasonable timescale and across a useful geographical set of developed countries, at the data on mathematical attainment, here’s the headline:

‘Scotland’s young people do better at mathematics than those in the rest of the UK as a longer-term decline in pupil attainment in maths and arithmetic seems to be slowing and levelling out’

Share on FacebookTweet about this on TwitterShare on Google+Email this to someone

24 COMMENTS

  1. Well I am glad it is “slowing down and levelling out” John .I have three grandchildren in secondary education and one soon will be . Quite honestly I feel their primary education was very hit and miss as far as maths and english is concerned . Some of it I feel is down to teaching staff , are they all finding it hard to cope ? . The children have all have so far managed to do well enough , but I would say it has more to do with their parents determination to keep them on the right path more than anything else . They all went to a school that ten years ago was the best in the area , I would not give it that accolade now . I hate to think what happens to the children whose parents take no interest and who come from one of the many deprived areas of Scotland . This big ship is going to take a lot of turning I fear .

    • Sorry to hear that. I’ve been happy with the experience my 4 had from 1989 till the present. Maybe too much pressure recently on the fourth to get enough highers for uni entry. Bit of an exam factory.

    • Your comment is the sort of personal, anecdotal stuff that lots of people love to go by, to the point where they will denounce the results of objective research covering large numbers of pupils over an extended period of time.

      i.e. It doesn’t help.

      If your grandbairns don’t know what i.e. means but are familiar with l8r, btw, lol, omg etc – that’s because they are, among other things, two generations apart from yir guidsel.

      wtf – imho, life goes on – etc etc etc. Ken what ah’m sayin’.

      • SCITE! I was a member of the Standing Committee on Initial Teacher Education back in the 80s so probably guilty for everything.

  2. I agree with John W that education is a patchy affair but my children all went through their secondary year long before the SNP was in power and it was the luck of the draw if they got decent teaching then. There is no way the gap in attainment will be levelled out overnight and as John W says, much has to do with the support children receive from families rather than the school system. But these flashy headlines that fail to report on a bigger picture are detrimental to improving standards, easy to blame a political party but ignore systemic social/economic failures.

    Another problem is trying to judge a new system like Curriculum for Excellence against traditional education standards. Old time education involved learning by rote, with a rigid focus on certain aspects of learning including maths and English. Looking way back we learned to parse and analyse, use cose and sine tables but with little emphasis on a broad curriculum that encouraged questioning and innovative thought. This was maybe ok in a world where you got a job and could expect to have it for life, but today’s children need to be much more flexible and have many skills to help them adapt to multi optional career choices. They also need very different skills – we already have calculators and computers which take away from the need for a fair bit of traditional maths/literacy but coding, texting and other newer strategies are now incorporated in the school curriculum.

    Since ancient Greek/Roman times adults have deplored the drop in standards of education for the younger generation but their measures are based on their experiences of the world. A better measure of the effectiveness of education is the onward trajectory of school leavers – are they fitted for today’s world? And that measure must take into consideration the societal factors that support or hinder their education. All in all I think the Scottish Government has at least attempted to address education issues with a view to ensuring all children get better chances without imposing a top down curriculum with only winners or losers.

    If only we had more journalists willing to go beyond a SNP bashing headline. The Scottish Government is not perfect but at least they seem to be trying to right some wrongs and the BBC lose a lot of credibility because of their imbalanced reports

  3. Another forensic deconstruction of BBC garbage. When are we going to get some journalists who are both literate and numerate so that they are able to read, understand and interpret surveys and research and know something about validity, reliability, cause and effect? Why, some of them, I’m told, have even been to university.

    One of the main factors in the “attainment gap” is parental income – or poverty. Yet, so many of the prescriptions are about helping “poor” families, parents and children to be “better people” – better attitudes, better aspirations, better parenting skills. But rarely (never?) “better off”. In other words, reduce inequality and at a stroke we could make things “better”.

    It’s the reason the rich send their children to fee-paying schools: money buys you successful educational attainment, as well as entry to the network of top jobs.

  4. Absolutely correct. You cant improve a complex society without changing it holistically. Tinkering with one element such as education or health in a disconnected way without addressing wider inequalities will not succeed.

  5. Thanks for that counter balance. This really focuses on an issue that frustrates me. I know the BBC will never be fair to the SNP government, but why oh why do the trade unions in education not step up to the plate and defend their members? They have traction. Your info should be spread far and wide. Give them it and tell them to get the finger out. (That’s an academic term when I grew up!)

    Many thanks for all your work.

  6. Living in central London, child of 10 at local primary.

    Total exam factory. Pressure is immense on kids, or non-existent where parents ‘dont care’. Pressure resulting in children not sleeping etc.

    Quality of teaching is good, but they are forced to teach to pressure. It so wrong, its hard to express my anger at it. Every parent I know hates it. I seriously hope Scotland does not go this route, and I can imagine that problems are exacerbated mostly in poor areas with our imposed austerity.

  7. Thanks Bert. Really sorry to hear this. Luckily the 10 year-old has you to help them keep it all in perspective/proportion – encourage, support, praise but always show complete disinterest in the performance of others.

  8. In a more general assessment, I remember you have complained in the past when so-called “experts” have appeared either in print, sound or vision, to give their opinion, heavily biased towards S.N.P baad of course. As you correctly pointed out, these people were simply not qualified to offer opinions, and now the situatuon has deteriorted where the M.S.M promotes all sorts to voice their particular views, as long as it fits their pre-determined agenda.
    I, and I know many others, consider the establishment controlled media our most dangerous opponents, because despite their declining infuence, they still, through their dominence of what we see, hear, and read, hold great sway over a great number of people.
    In the particular report you highlight, as I am sure you are well aware, there is no chance of your balanced assessment seeing the light of day, at least in the M.S.M.
    On the other matter you mentioned, and I’ve said to other notable contributors on other independence supporting websites, please don’t stop. The information you impart to us, and I am just a foot-soldier, is invaluable, especially when canvassing on the doorstep. We have the council elections coming up, so we need all the onformation we can get in order to get our message across, and correct the false flags promoted by the unionist media.

    • Thanks Alex. Notably on recent education topics -HE entry and attainment in schools – they haven’t even managed to get an expert of any kind to say what they want.

  9. I concur 100% Alex.Here’s a thought..do you think STV would be interested in airing a documentary entitled..’Is Auntie biased against the SNP?’……..Since they’re neighbours @Pacific Quay..probably not…but nothing ventured nothing gained?

  10. In my whole life of 67 years I have never heard any different from any political party about education, they all use it as a bat to beat the other side with and nobody ever wants to tell the whole truth that our kids are well fed lazy useless articles who won’t move themselves from the front of the Telly or get out of their beds

    Eh, or is that the parent argument from always

    Folk just love to blame “The government” but come on, we all know it’s the bone idle 3 month holiday fancy car teachers who don’t want to work for their overblown salaries

    Or is that an old argument, at my age I’ve heard them all

    If we didn’t live in a country where the media somehow thinks it should be in charge maybe, just maybe the powers that be wouldn’t have to go running around like headless chickens over the media screeching about a percentage point here or there and be able to get on with doing the best job they can in a bit of peace

    Folk who demand answers to meaningless manufactured problems should expect to have their noses punched
    Leave the experts to work out what’s needed and the government to find a way to fund it and shut the media’s gubs about it

    It’s not helping!

    • It’s deja vu all over again as I heard a Fox News reporter say recently. Having been all of the ‘articles’ you correctly describe as lazy and useless, I can only thank Dog for the whisky.

  11. PISA rankings for Scotland plummeted under the Lab-LibDem tenure.
    This has been dramatically slowed to near level or in some cases improvement since the SNP took over.

    Some of the more recent ranking table placements have been distorted somewhat by the increase in number of Chinese education authorities which demand a student workload which would be unacceptable in the West and which has as yet unknown consequences on student well being.

    On further education, Scotland has the most universities in the top 200 of any country in the World on a pro rata basis.

    Scottish universities have the highest project roll out (commercialisation) in the world, pro rata.

    With approx. 0.07% of the World population, Scottish university produced papers are cited in 1 in 50 of all papers produced in the World.

    And still no student tuition fees.

  12. Education is “leading out knowledge”.

    This phrase suggests there must be something there for the teachers to stimulate and work with in the first place. During a child’s life what’s in them or what they wish to do waxes, wanes, rises and falls, etc.

    It not necessarily what the child gets from school, it’s what the child brings too the school that”s more important, this is where privelidge comes to the fore.

    Yes there are exceptions where kids from poor.backgrounds that succeed, but they are exceptional people who battled invisible odds and won.

    Conversely how may kids from privileged backgrounds fail?

    In the US in 50, 60 and 70 they spent billions trying to raise black attainment to that of whites, ithe policy failed. They would have had a better outcome if they had educated the mums and dads.

    Teachers in Scotland have been a fickle bunch, industrial action here there, threatened now,next year or when new examss come in, we’ve too much work, etc, etc but seldom had the courage of their convictions and gone on strike.

    Swinney will will ensure he takes them with him, the ball is in their court, it’s time for them to get stuck in!

    • !980’s – Scottish teachers possibly the only group of workers to take on the blessit Magi and win.

      My memory is that said Magi wanted to close the schools down – it didn’t happen because the public outcry would have devastated the Scottish Tories – but if it had, teachers wouldn’t have lasted a couple of weeks because they don’t have the same sort of community as miners or steel workers – courage doesn’t come in to it.

      But a brilliant union leader, whose name I forget, organised a strike whereby schools were closed on a rolling basis and teachers not on strike contributed to a fund to make up for lost wages for those on strike. They could have kept going for ever, and public opinion was on the teachers side.

      Her blessitness eventually gave in, but had to have her pound of flesh (nearly half an armful) in the form of making compulsory things that teachers had previously done on a voluntary basis – and nearly ruined Scottish school rugby in the process.

      https://www.pressandjournal.co.uk/fp/news/politics/westminster/434988/call-to-let-scots-stew-during-schools-row/

LEAVE A REPLY

Please enter your comment!
Please enter your name here