By Dr Nina Hood

Better knowledge and use of data in schools are essential if teachers want better to understand their teaching, improve their students’ learning, and understand the intersection between the two. For Dr Aaron Wilson, data provide insight into the black box of teaching, which is positioned between and mediates the input of students’ learning opportunities and the output, student learning outcomes. In his seminar last week, Aaron positioned data as a critical component for teachers (and schools) to better understand their impact, to drive continuous improvement, and to support early intervention.

The day-to-day reality of teaching means that teachers typically have a “zoomed in” view of their students and their classroom practice (something Professor Graeme Aitken has previously discussed). Teachers’ work is focused in the moment, responding to the ever-changing conditions of their classrooms. This means it often can be difficult for teachers to get the perspective they require to look objectively at their classroom practice. Something compounded by teachers’ self-reports being notoriously unreliable. It is perhaps this zoomed in view which explains why education so frequently suffers from “solutionitis”. That is, jumping very quickly from problem definition to potential solution, often without fully understanding the root cause(s) of the problem and the range of drivers.

Data use can provide one means for enabling teachers to take a more “zoomed out” perspective. As Aaron suggested, it is by zooming out that it is possible to discern important patterns in teaching and learning with some measure of objectivity. However, this must be complemented by a continued, deep commitment to, and knowledge of, the local context. It is important not to divorce high level data analysis from the practical, situated knowledge of educators.

In this conception, data are not intended to be used as a high-stakes accountability measure or to make overly simplistic judgements about teaching and teachers. Nor should data be used to focus on improving test scores without also improving a full range of valued learning outcomes. Instead, they enable rich understandings of patterns of teaching and of learning, and in particular how and why patterns of teaching contribute to current patterns of student learning.

To enable this zoomed out to zoomed in approach to data use, it is important that the data being collected and analysed are sufficiently varied to capture the multifaceted nature of teaching and learning, whilst retaining the level of specificity needed to make meaningful judgements. Data must focus on achievement levels as well as on rates progress. And when analysing data, it is critical to move beyond measures of central tendency to also explore variance (Todd Rose’s The End of Average provides an interesting read on the limitations of relying on averages). That is, it is necessary to interrogate data in ways that enable us to understand if some students or some groups of students are making greater gains than others.

For Aaron, above all else, the starting point of data use in schools should be identifying valued learning outcomes. These must include, but crucially also must go beyond, measures of academic progress. Intermediary outcomes and affective measures are also important for schools.

However, the identification of valued learning outcomes and the collection of data on these is only the first step for schools. The real power of data lies in how those data are used to make changes and to drive improvement in teaching and learning. Schools need to be able to interrogate their data to identify the aspect of student learning that will be catalytic. This not only requires the ability to effectively analyse data but also the deep knowledge of the students and strong pedagogical content knowledge to enable the identification of next steps.

Aaron suggests, it is unreasonable to expect schools to do this on their own. In fact, effective data use in schools should be a collaborative undertaking, involving schools working with expert partners to help them with data collection and analysis, and to devise, monitor and iteratively improve next steps. “The stubborn and persistent problems that are most worth inquiring into can seldom be solved by one person. Good inquiry requires teams, leadership support and challenge, external viewpoints, external expertise, and moral support” (A. Wilson, 2019).

This poses something of a challenge in the New Zealand context. Our current system is not set up to readily allow the types of research-practice partnerships that Aaron suggests are most effective for supporting data use in schools. We lack the broad infrastructure, including external experts with the requisite skills and knowledge, to work with schools in this way.

If the effective use of data (broadly conceptualised – something I previously have written about here and here) is central to the improvement work of schools, it is critical that we question how we develop the support, expertise and resource (including providing teachers with the necessary time) to do this work?