Media analyst John Robertson takes issue with the BBC’s coverage of a major education story
‘Critics describe a steady decline in pupil attainment in maths and arithmetic as ‘appalling!’ (Reporting Scotland, 31st May 2016)
This headline, on the Scottish Survey of Literacy and Numeracy 2015 (Numeracy), is simply wrong. The Scottish Survey of Literacy and Numeracy is a Scottish Government initiative started in 2011/12. There is no statistically valuable evidence of educational change to be drawn from data collected every two years, with very low single-digit changes between collections, and covering such a short total time as four years. Try submitting an undergraduate university dissertation proposal based on this and watch the supervisor laugh till he hurts.
I’m the one accusing BBC Scotland of decline and I have the credentials to do so, having published in the top, peer-reviewed, media-analysis journals. The ‘critics’ accusing the Scottish Government’s performance on educational attainment are the usual suspects, close to the BBC’s heart, members of the Tory and Labour parties. These politicians have little credibility having never published in peer-reviewed educational research journals and, in most cases, never taught a class of 30 wean-bairns. I’ve done both. The teachers’ union, the EIS, offered a more intelligent analysis, linking these figures to wider austerity, but only appeared on STV. No serious academic researcher was daft enough to join in the feeding frenzy. Actually, it was no academic researcher at all.
Here are the actual figures from the full report, available online, in graph form:
The above graphic is from Chapter 3: Numeracy attainment over time at: http://www.gov.scot/Publications/2016/05/2836/4
Here is how this graphic is interpreted on the gov.scot site and indeed should be interpreted:
The proportion of P4 pupils performing well or very well decreased between 2011 and 2013, and again slightly between 2013 and 2015.
The proportion of P7 pupils performing well or very well decreased between 2011 and 2013 but remained constant between 2013 and 2015.
The proportion of S2 pupils performing well or very well remained stable between 2011, 2013 and 2015.
I’m not suggesting this is good. It is evidence of some ongoing decline but also of a deceleration of that decline or even of its flattening out. The figures from deprived areas are worrying. However, they are not as described, ‘appalling’.
Stage 2011 to 2013 2013 to 2015 2011 to 2015
P4 Lower in 2013 Lower in 2015 Lower in 2015
P7 Lower in 2013 No difference Lower in 2015
S2 No difference No difference No difference
Table 3.3 ‘Most deprived category’ (pupils from the most deprived 30% of datazones)
The change from 2013 to 2015 (middle column) is important because it may reflect more recent changes in policy which we need to be able to evaluate. These suggest a flattening out of a decline. Now if that decline is one which goes back well before 2013, it’s stalling, decelerating or flattening might be a sign for optimism of the type not familiar to BBC Scotland if they think it might reflect well on the SNP. The good news is that we can go back much further, to 2000 AD, deep into the long days of New Labour Scotland Branch rule and, presumably, better attainment? These data from PISA (Programme for International Student Assessment) cover a broader range of mathematics skills. See this from PISA 2009 which reveals a decline going back at least to 2000.
Year Mean Score
2000 533
2003 524
2006 506
2009 499
These figures can be found in: http://www.gov.scot/Publications/2010/12/10141122/16
I’ll update on PISA 2012 below.
So, what we have here is evidence of a longer decline perhaps caused by New Labour policies and being slowed, if not stopped, by the Scottish Government’s actions in the last few years. Remember, changing educational outcomes takes time, decades even.
You’ll remember, only last week we heard, erroneously, that disadvantaged young people in Scotland were actually less likely to enter higher education than disadvantaged young people in England. See my NN piece last week for more on this. BBC Scotland does like a comparison between our education systems or our health services. Let’s see if PISA had any such comparisons to help us all contextualise and to update from PISA 2009, the Scottish data.
‘Scotland was the best for reading and maths of the UK nations, ranked 15th and 21st (out of 67 developed countries)
That’s a BBC headline at: http://www.bbc.co.uk/news/uk-wales-11930257
Here’s a bit more from the actual PISA 2012 report with page numbers in brackets:
Scotland had the smallest percentage of pupils working at the lowest levels in all three subjects and their low achievers scored more highly in all subjects. (83)
In all subjects, Scotland had the lowest percentage of pupils at Level 1 or below (83)
The highest attainment for mathematics was in Scotland (84)
Surely, just a wee smile is justified here? Have some more:
The distribution of performance in mathematics showed that there was some degree of variation around the mean score for mathematics in all countries, as would be expected. The size of this variation indicates the extent of the gap between low and high attaining pupils. The OECD average score was 301 points. The range was wider than this in England and Northern Ireland and narrower in Scotland and Wales. The highest difference of 316 was found in England. (87)
So, in the UK context, Scotland has a narrower attainment gap than England. Further, clear evidence of this is:
Scotland had the lowest percentage of pupils working below Level 1 in mathematics (4.9 per cent). This compares with the OECD average of 8.0 per cent. In England and Northern Ireland the proportion of pupils working at the lowest level of proficiency in mathematics was close to, or the same as, the OECD average (8 and 8.6 per cent respectively). (88)
Scotland had the narrowest range of attainment and the scores of their lowest achieving pupils were much higher than those in the rest of the UK or the OECD on average. (99)
The above can be found in more detail at:
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/299658/programme-for-international-student-assessment-pisa-2012-national-report-for-england.pdf
‘They’ve had enough’, you’ll be saying but remember this is the England we so often find BBC Scotland fawning over and throwing at us deluded independence supporters, so here are more examples:
Young people in England are the most illiterate in the developed world with many students graduating with only a basic grasp of English and maths, an in-depth analysis by the Organisation for Economic Co-operation and Development has found.
The OECD report rated English teenagers aged 16 to 19 the worst of 23 developed nations in literacy and 22nd of 23 in numeracy..
http://www.ibtimes.co.uk/young-people-england-have-lowest-literacy-levels-developed-world-says-oecd-1540711
Finally, returning to the PISA reports, to further put the Scottish trend (pulling out of a long decline?) in context:
‘Of the 64 countries and economies with trend data between 2003 and 2012, 25 [only] improved in mathematics performance.’ (4)
https://www.oecd.org/pisa/keyfindings/pisa-2012-results-overview.pdf
So, having looked more thoroughly, over a reasonable timescale and across a useful geographical set of developed countries, at the data on mathematical attainment, here’s the headline:
‘Scotland’s young people do better at mathematics than those in the rest of the UK as a longer-term decline in pupil attainment in maths and arithmetic seems to be slowing and levelling out’
