Relying on Narrow Metrics to Measure Accountability is no Way to Ensure Children get the Education they Deserve
In the last half century, the practice of reducing performance solely to easily understood statistical models has spread its tentacles into public sector provision across the globe, says Fiona Millar…
- by Fiona Millar
What is the connection between the Vietnam war and today’s school system? It is a stretch of the imagination isn’t it?
However bleak we may feel about education policy at times, it can’t possibly be equated with one of the biggest foreign policy disasters of the last century. Yet, with hindsight, one aspect of the war proved to be an ominous foretaste of the world as we know it today.
At a time when the political and military challenges facing the American government were starting to feel overwhelming, the then Defence Secretary Robert McNamara, a former accountant, introduced the idea of an enemy body count as a reliable index of progress towards winning the war.
Many historian hours have been spent pouring over the subsequent trajectory of the war but one widely held conclusion is that what became known as the ‘McNamara fallacy’ – relying on metrics at the expense of other less easily measurable factors – could be deeply flawed.
Perverse incentives
And yet this is a lesson that is still being learned. In the last half century, the practice of reducing performance solely to easily understood statistical models has spread its tentacles into public sector provision across the globe.
It is over 20 years since league tables, inspections and performance objectives based almost exclusively on data – in other words test and exam results – became de rigueur in English schools.
Yet every week seems to bring another example of the unintended consequences of this approach, such as exam cramming, teaching to the test, narrowing the curriculum, off rolling of particular pupils and admissions ruses to keep certain students out in the first place.
Encouraging schools to compete with each other according to narrow metrics has brought with it perverse incentives that further disadvantage the most disadvantaged.
Over reliance on data almost certainly discourages risk taking and innovation and, when it leads to a toxic working culture, has a knock-on effect on staff recruitment and retention in some of the most challenging schools.
Desire for change
The idea of standardised numerical data as part of a picture to judge performance isn’t a bad one; those of us who are old enough to remember the pre-league table era know that too many children were allowed to fail in schools that weren’t accountable enough and had no clear objective means of judging success (and failure).
But there is a growing sense that the benefits of oversight by data has gone too far. The NAHT has just produced a report, Improving School Accountability, that suggests at the very least data should only be used to compare schools in similar circumstance and should be viewed over a three-year period.
The Chief Inspector of Schools is rumoured to want to change the focus of Ofsted away from exam cramming to a broader vision for education.
And in his excellent new book The Tyranny of Metrics, American academic Jerry Z Muller spells out the causes and dangers of over-reliance on data and helpfully contextualises the Vietnam body count decision and other formative moments that brought us to this point.
It feels like a head of steam, so what is the barrier to change?
More trust
The desire for accountability is understandable, but the belief that only ‘that which can be measured is worth measuring’ is questionable and the subtext, that professional knowledge and judgements can’t be standardised therefore can’t be trusted, is plainly wrong.
For government to sanction a different approach – for example the inclusion of less easily measurable variables like breadth of curriculum, pupil wellbeing and even parent satisfaction – it would need to trust teachers more and stop worrying that we would slip back to a time when schools and young people were allowed to fail.
Whether you look at off rolling, exclusion, gaming the curriculum, cheating or failure to recruit enough good teachers in the most challenging schools, many young people are being failed today, even after decades of this type of narrow accountability.
It would be a bold politician who made that first move, but the current situation demands that.
Fiona Millar is a columnist for The Guardian and a co-founder of the Local Schools Network; for more information, visit fionamillar.com or follow @schooltruth.
