Educational research – How to tell the good from the bad
Harry Madgwick and Kirstin Mulholland highlight the pitfalls educators should be wary of when weighing up the merits of education research
Educators and leaders are increasingly turning to educational research evidence to help identify promising approaches and practices that have the potential to make a difference to pupils and communities.
However, we know from our experiences of working in schools and teacher professional development that while many products and resources are now badged as being developed based on ‘evidence’, finding the time to access, engage with and actively question said educational research evidence can be extremely difficult.
Leaders are liable to be bombarded with information regarding programmes and resources that make all manner of impressive claims. This is often courtesy of companies and organisations with a vested interest in selling those very same products or services.
So, when faced with the vast quantities of evidence available, how do we know which sources are trustworthy and relevant to the problems we are trying to solve?
The Education Endowment Foundation’s concise guide to using research evidence includes information on what educational research evidence is – and is not – as well as the advantages and potential limitations inherent to different types of research.
We hope the guide is able to help those involved in school improvement, and in the design and delivery of teacher development, to better reflect upon the role educational research evidence can play in educational outcomes and CPD activities.
Above all, we hope it will prompt educators to consider what kind of evidence they’re using, as well as why they’re using it, and the ways in which they ought to apply it. This way, we can harness the collective knowledge of what has worked in the past to make a meaningful difference to teaching and learning in future.
Education research – a user’s guide
Build up a rich evidence picture
When looking at any area of educational research, it’s important to always consider multiple studies from a range of sources so that you can identify any common themes and trends. Try to avoid ‘cherry picking’ educational research that confirms your existing beliefs, and instead take a broader view of the evidence base.
Drawing from systematic reviews and meta-analyses that combine multiple previously published studies can help with this. For an example, see the EEF’s guidance reports, evidence reviews and Teaching and Learning Toolkit.
Look for variation in findings
Remember that the devil really is in the detail. When engaging with educational research evidence, be watchful for any variations in findings across different studies and what the reasons behind those variations might be. For example, are there any groups of pupils for whom a given approach seems more, or indeed less successful? If so, why?
Focus on the ‘how’ – not just the ‘what’
Look carefully at how someone has implemented a specific approach before deciding on whether you could apply the elements that make that approach effective in your context. The EEF’s implementation guidance report provides some practical recommendations that may be useful when engaging with such questions.
Maintain criticality
Be open to new ideas, but adopt a critical approach to the claims people make. Look out for warning signs, such as claims which seem too good to be true, or vaguely referenced, non-cited mentions of ‘impact’.
The EEF concise guide contains a list of red flags to be mindful of when it comes to the reliability of evidence. You can remember this using the mnemonic CLAIMS:
- Conclusions
- Limitations
- Applicability
- Independence
- Methods
- Sample size
Harry Madgwick is the EFF’s research and policy manager; Kirstin Mulholland is its senior associate for school engagement and evidence use – for more information, visit educationendowmentfoundation.org.uk
