Evidence and the real world
There is a growing appetite in Australia for more evidence-based policymaking in education. In particular, policymakers are often called on to use evidence of ‘what works’ when designing policies to improve teaching in the classroom.
By Jordana Hunter
This is a welcome development. Policies based on robust evidence of what works – as opposed to particular interests, ideological beliefs, popular fads or historical precedent (the ‘this is how we have always done it’ way of thinking) – have a greater chance of lifting outcomes for students.
In an ideal world, all education policies would be based on evidence of what works. But, here in the real world, there are many reasons why this can be hard to do. Indeed, evidence-based policymaking can be a difficult, at times frustrating, endeavour.
Government policies are often complex and multilayered. They can be hard to change and even harder to implement. Often the evidence is murky or contradictory. Sometimes researchers genuinely don’t know much at all about what works for a particular issue or context. Still, political imperatives often require action – pronto! Even if researchers do have all the answers for a particular problem, ‘best practice’ can change over time.
In reality, evidence-based policymaking requires an ongoing process of experimentation, evaluation, refinement, improvement and, as uncomfortable as it often is, failure.
What’s more, evidence-based policymaking often involves recognising the limits of the existing evidence base and having the courage to engage in reasoned analysis about the best way forward, even where ‘hard evidence’ is lacking. This means carefully thinking through tough policy problems and the risks and benefits of different responses.
In fact, the ability to pin down and interrogate a proposed policy’s ‘theory of change’ – including the particular steps or mechanisms that can be expected to connect policy ‘inputs’ to a desired set of ‘outputs’ and ‘outcomes’ – is a key tool in the evidence-based policymaker’s toolkit.
While evidence of ‘what works’ is a great place to start, policymakers can improve policy design by cultivating a broad appetite for ‘evidence’ beyond the existing ‘what works’ research. Four other types of evidence are particularly important.
Evidence of what doesn’t work. Some of the most effective forms of evidence-based policymaking involve refining or scrapping existing policies that encourage practices that are no longer supported by research evidence. This can involve tough decisions, but the pay-offs can be significant.
Evidence of current practices in schools and classrooms. Knowing what is currently happening inside schools (opening up the ‘black box’) is essential for policymakers to understand the types of challenges teachers and students face, prioritise among these, and design sensible, well-targeted policies in response.
Evidence about policy implementation. It is one thing to be familiar with the ‘what works’ evidence base. It is quite another to know how to design and implement policies that ensure evidence is put into practice in schools. Incorporating what is already known about effective implementation approaches is essential for good policy design. Policymakers can also build this evidence base by evaluating implementation processes, not just policy outcomes. Policies that have no chance of being implemented help no one.
‘Missing’ evidence. Given the limitations of the ‘what works’ evidence base, policymakers should think hard about the questions they most want answered and commit to finding out more. A process of disciplined innovation, including small-scale pilots that are evaluated rigorously, can help fill in the gaps in our knowledge.
Committing to evidence-based policymaking doesn’t guarantee we can or will solve all of the hard problems in school education. But it increases the odds that we will get a little closer to achieving the things we really value. That makes it worth the effort.
This article appeared in the Dec 2021 edition of Nomanis.
This article originally appeared on the Australian Education Research Organisation (AERO) website.
Dr Jordana Hunter [@hunter_jordana on Twitter] leads the School Education Program at the Grattan Institute, which is an independent public policy think tank. Jordana has an extensive background in public policy design and implementation, with expertise in school education as well as economic policy. She has co-authored a number of Grattan reports on school education focusing on education strategy and teacher professional learning. Her particular research interests are in approaches to designing government policies to support quality teaching and evidence-based literacy and numeracy instruction. Jordana has a PhD in education system reform from the University of Melbourne.