My objective today is to put words in a few prominent researchers’ mouths—or better yet, their paper, “Learning that Lasts: Unpacking Variation in Teachers’ Effects on Students’ Long-Term Knowledge.” Benjamin Master, Susanna Loeb, and James Wyckoff have posted online a “preliminary draft,” which no one is supposed to cite. Blogging, I assume with all online content, is fair game. This paper is terrifically important. I just want to see a draft that more fully discusses the many factors that contribute to teacher effects.
Let’s start with why this paper is worth your time: It’s a blockbuster for those worried about the negative consequences of annual high-stakes test-based teacher evaluations. Looking at the long-term impact of teachers with high value added, the researchers conclude:
Evaluation and accountability systems may incentive educators to focus excessively on short-term tested outcomes in ways that are not ultimately beneficial for students…. Collectively, this body of evidence demonstrates that teachers’ instructional practices can influence their short-term value-added performance in ways that do not correspond with long-term success for students…. Overall, our results demonstrate that teachers’ effects on students’ long-term skills can vary substantially and systematically, in ways that are not fully captured by short-term value-added measures of instructional quality.
We clearly need education policy to incentivize (or at least not impede) meaningful educational gains, so I hope policymakers will heed this research.
To increase the odds that they will heed it, the paper needs one quick little addition: more forceful acknowledgement that teachers’ effects are influenced by many factors. Several federal and state policies could be explored as means of positively influencing curriculum and instruction. This is not simply a teacher issue. It is a standards, curriculum, assessment, accountability, teacher preparation, professional development, leadership, and resource-allocation issue.
Is it really these researchers’ job to remind readers of the broader context? No. It’s just something that, given the importance of the issue, I’m hoping they’ll want to note.
When it comes to teacher effects, context matters.
Reading this paper, it’s easy to get swept up in thinking the teacher makes all the difference. For example, the more academic knowledge teachers have, the more they seem to infuse that in their instruction, to great effect:
The within-subject value-added persistence of ELA teachers who attended a more competitive undergraduate institution is significantly and substantially higher than that of teachers who attended a less competitive institution…. Differences in persistence are similarly large when comparing teachers whose SAT Verbal exam scores or LAST licensure exam scores are in the top third of the teacher distribution, in comparison to lower-scoring teachers. In both cases, higher scoring teachers show greater persistence…. It is notable that our teacher ability characteristics predict large differences in ELA teachers’ value-added persistence, even though they are not themselves correlated with teachers’ short-term value-added effects.
One might be tempted to think these direct teacher effects are simply teacher-quality issues. But nothing in education is so simple. Ask yourself: what’s likely to make a person with the potential to have lasting effects want to be a teacher for years to come? Rigorous standards and engaging curriculum, meaningful assessments that support instruction, accountability policies that don’t incentivize test prep, academically demanding preparation programs, tailored professional development, helpful leaders, etc.
Much of the paper is devoted to examining potential student- vs. teacher-level drivers of the variation in teachers’ long-term impact. That, obviously, is a key question—I just want to see more acknowledgement that the “teacher effects” are federal-, state-, district-, and school-policy effects. Here’s the heart of the research:
Observable student characteristics related to their socio-economic status or prior ability also predict substantial variation in their ELA teachers’ value-added persistence. The persistence of achievement gains coming from having an effective teacher is far lower for students who are eligible for free lunch, are black or Hispanic, or whose twice-lagged ELA achievement scores are below the mean…. These students may be receiving ELA instruction that is less focused on long-term knowledge, or they may be less skilled at acquiring or retaining long-term knowledge.
Ultimately, the researchers conclude the primary issue is likely “instruction that is less focused on long-term knowledge”:
We see evidence of the importance of instruction in the positive association between teachers’ academic ability and their contributions to students’ long-term knowledge. Even more compelling, we find that schools that serve more disadvantaged students or that hire fewer of these high-ability teachers have lower value-added persistence in ELA for all of their students. Students, regardless of their prior test performance, who attend schools with many low-performing students demonstrate lower persistence of the learning gains they achieve from having a high value-added teacher. The persistence in low-achieving schools is less than half the rate of that in other schools. These findings provide evidence that instructional quality is a key driver of the variation that we observe in value-added persistence, and that school-level curriculum or instructional norms may foster differences in instructional quality. Unfortunately, we are unable to directly observe the instructional practices of teachers or schools in our sample. However, in light of prior research on educators’ responses to high stakes accountability pressures … one plausible explanation for our findings could be that schools serving lower performing students systematically prioritize gains in short-term tested achievement in ways that detract from teachers’ focus on long-term knowledge generation.
As I’ve said, there’s a whole lot beyond “school-level curriculum or instructional norms” that “may foster differences in instructional quality.” The authors of this paper know that—and it’s certainly not their fault that many policymakers need to be reminded. But they do. And if more policymakers get the message that we have a multifaceted, highly complex problem to address, perhaps more desperately needed research dollars will be provided and more varied policies will be piloted.