By Dr Nina Hood

Last week Simon Collins wrote a series of articles in the New Zealand Herald exploring “modern learning” in New Zealand schools. In one article he wrote: ‘schools are gambling with our children’s future – taking a punt on new ways of teaching, for which there is not yet much hard evidence of success’. I don’t necessarily disagree with Collins’ assessment. However, it did make me question how might we develop the ‘hard evidence’ if we do not first ‘take a punt’?

Until we implement an innovation, it is challenging to undertake the research that will produce the data and evidence to understand the impact it is having and to enable us to assess whether it is actually leading to improvement. Consequently, it is possible that a change may in fact lead to worse outcomes or negative impacts but until we implement it, we will not know for certain. Given this bind, how should we balance the need to be innovating and experimenting to ensure that we are offering the best educational opportunities and outcomes with the understanding that implementing a change that does not lead to improvement will impact the learning and lives of young people?

Last week I wrote of the need to connect research and innovation in education, so that we are generating the evidence to determine the impact (and implications) a change is having. Missing from that account was adequate reflection on the challenges associated with this work. Below I explore five such challenges.

Challenge one: the lab versus the classroom

Unlike in other sectors where researchers can first trial new approaches in the controlled environment of a lab, such an approach is arguable less useful education. Educational psychologists and cognitive neuroscientists do continue to run experiments in labs, and increasingly their findings are making their way into classrooms and shaping teachers’ practice. However, experiments in labs risk reducing the complexities of the ‘hot-action’ context of the classroom. Teaching and learning as an interactive and dynamic enterprise shaped by a myriad of nested factors – individual, within-school, out-of-school and systemic – are not fully captured.

Challenge two: short-term versus long-term impact

In recent years some in education have sought to implement the rapid learning cycles being developed in other sectors. Variously known as lean, agile or design-based methodologies, all emphasise the need to learn fast through repeated evaluation and iteration. There definitely is much education can learn from these approaches. If something is not working, it is crucial that steps are taken to address this as soon as possible. Similarly, if something is working well, understanding for whom it is working, and why, can lead to even greater impact.

However, education is not only concerned with the short game. It also is necessarily to understand the long-term impact of particular approaches and practices. This is particularly important given a number of studies showing that effects tend to fade over time. This need to understand long term impacts, however, poses a number of challenges. Findings will not be known potentially for decades, and in a majority of cases, research does not continue to track an intervention long-term. Decision making timelines in education typically are short, meaning innovations often do not have time to be fully embedded or evaluated, before new changes are implemented.

Challenge three: intended and unintended consequences

Too frequently, educational research fails to fully consider (or report on) unintended consequences of an initiative or practice. As Professor Yong Zhao has argued, when implementing a new practice, time and resource is necessarily diverted away from something else, which may result in unintended consequences, such as the narrowing of the curriculum. Furthermore, some approaches may work better for certain students than others. When evaluating an approach, it is crucial that a wide range of outcomes are considered.

Challenge four: what and how to evaluate

Rigorous evaluation in education faces a series of challenges. What is measured should stem from the desired outcomes of the particular intervention or approach, which in turn should be informed by broader questions as to the purpose of education. However, frequently in education we default to measuring what can easily and reliably be measured, which tends to be a narrow set of outcomes related primarily to literacy and numeracy. To rely only on these scores as a proxy for learning and educational quality more generally, is highly problematic. Measurement is further complicated by the wide range of factors that impact on learning and outcomes, which makes determining causal effects incredibly challenging in education.

Challenge five: consistency at scale

Few educational innovations, initiatives or programmes have retained impact at scale. Education is hampered by issues of transfer (disseminating knowledge from one person to another) and transposition (disseminating knowledge from one place or context to another). There is a growing body of thought that suggests that education should be less concerned with treatment fidelity and more focused on process fidelity. That is, the process of knowing how to improve and the conditions, including the learning required, to enable improvement to occur across contexts.

Given the five challenges above, how can education address this innovation bind? Firstly, it is important that any change in education is informed by the existing knowledge base. We know a lot (though still not enough) about how people learn and the conditions that best foster the broad range of learning outcomes our education system aspires to. This evidence should underpin the creation of new directions and initiatives.

Any change or new initiative should start small, be rigorously evaluated and iteratively refined, before being scaled more broadly. And scaling efforts must also be evaluated to understand what works, for whom and in what contexts. For this type of evaluation to occur consistently, it is necessarily to develop the systems and processes, and build the capability among those working in education, to support such continuous improvement work. It is critical that schools do not bear this burden alone. Instead, it is necessary to establish multi-stakeholder partnerships.

Finally, innovation in education must have a deep commitment to addressing the seemingly intractable problems facing our education, and most importantly, the entrenched inequities in educational opportunities and outcomes.