Education Policy Makers Need to Understand – Data Does Not Equal Knowledge

by David Didau

Share

DOWNLOAD A FREE RESOURCE! Year 6 science – Animals including humans assessment worksheet with answers Download Now

PrimaryAssessmentScience

Schools are awash with data but do we know any more about how children are performing, how likely they are to achieve particular targets or what’s preventing them from making progress? All too often the answer is no. The problem can be simply summed up as data ≠ knowledge.

There’s a lovely video on the internet of Jamie Oliver, showing a group of youngsters what goes into chicken nuggets.

He whizzes up a mixture of skin, bone and ‘horrible bits’ and explains that manufacturers squeeze this revolting goop into a machine that removes all the crunchy stuff and leaves you with a smooth paste.

At the end of the process he shows the kids some nuggets and asks, “Who would still eat this?” All their hands shoot up.

The data machine is similar.

We make up numbers based on a combination of hunch, bias and ill-conceived assessments, feed this slop into a machine which removes anything inconvenient and at the other end we get flight paths, progression models, age-related expectations and all sorts of other gibberish.

Who would still eat this?

The easy option

To understand what children’s performance tells us about how they likely they are to achieve, we need meaningful information, we need to know what happens to this information and we need to know what to do as a result of collecting and manipulating it.

Sadly, it’s much easier to simply chow down on reheated, reconstituted giblets than to do the hard work of cooking up some delicious, nutritious data from scratch.

Why, back at the very dawn of the information era, Victorian engineer Charles Babbage, inventor of the earliest computer, recounted being asked by various influential individuals, “Pray Mr Babbage, if you put into the machine wrong figures, will the right answers come out?”

He confessed that, “I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.” The same confusion of ideas is alive and well a century and a half later. In some ways we’re worse, as the modern data consumer doesn’t even realise we routinely feed wrong figures into the machine.

Real meaning

So, what’s the solution? First, we need to separate the notion that data and numbers are synonymous. We must get into the habit of asking what numbers actually represent. If a student is predicted to get a grade 5 in their Spanish GCSE, what does this mean? What will they be able to do, and what will they struggle with?

Sadly, in most cases, being predicted a 5 simply means better than a 4 but not as good as a 6. The numbers act as black holes, sucking meaning into themselves and warping our perceptions of reality.

Substituting numbers for labels is no better; we need to be clear about what such labels mean. If a parent is told their child is making ‘age related expectations’ what does this tell them? Are our expectations based on bias and stereotype? Undoubtedly.

This is an essential part of the human condition. We find it hard to juggle too many variables and find it so much easier to say this is like that. All we’re really saying is that this child is broadly able to do what an average of other children can also do. But what specifically are we talking about?

What next?

Then, what happens as a result of data having been collected and manipulated? Does it help anyone to know a child is predicted to get a grade 4 in GCSE maths? The acid test for any form of data collection is to ask whether having it will lead to better teaching.

How will this data help a teacher to understand what it is a pupil can’t do or doesn’t know? Too often this essential question is only obscured by relying on the data machine.

Crucially, teachers and school leaders need to know that no one is asking them to collect garbage data. Both the DfE and Ofsted have stated, unequivocally, that there is no such requirement for, in Amanda Spielman’s words, ‘byzantine number systems’. If schools choose to waste time in this way they have only themselves to blame.