Miami English teacher Roxanna Elden makes a compelling case for how “data-driven instruction” can be misleading and self-defeating. Writing at Education Next, Elden describes a nonfiction passage about owls on a practice test for the state’s FCAT test: Which of the owls’ names is the most misleading? Is it the screech owl “because its call rarely approximates a screech?” Or is it the long-eared owl, “because its real ears are behind its eyes and covered by feathers?”
Each question on the practice test supposedly corresponds to a specific reading skill or benchmark. “Teachers are supposed to discuss test results in afterschool ‘data chats’ and then review weak skills in class,” Elden writes. Like so:
First Teacher: Well, it looks like my students need some extra work on benchmark LA.910.6.2.2: The student will organize, synthesize, analyze, and evaluate the validity and reliability of information from multiple sources (including primary and secondary sources) to draw conclusions using a variety of techniques, and correctly use standardized citations.
Second Teacher: Mine, too! Now let’s work as a team to help students better understand this benchmark in time for next month’s assessment.
Third Teacher: I am glad we are having this “chat.”
Forget for a moment that people only speak like this after they fall asleep next to a pod. Here’s how Elden’s actual “data chat” went:
First Teacher: My students’ lowest area was supposedly synthesizing information, but that benchmark was only tested by two questions. One was the last question on the test, and a lot of my students didn’t have time to finish. The other question was that one about the screech owl having the misleading name, and I thought it was kind of confusing.
Second Teacher: We read that question in class and most of my students didn’t know what approximates meant, so it really became more of a vocabulary question.
Third Teacher: Wait … I thought the long-eared owl was the one with the misleading name.
Language arts teachers, Elden points out, “know that answering comprehension questions correctly does not rest on just one benchmark.” That may work for math, but, she correctly observes, “reading is different.”
“After students have mastered basics like decoding, reading cannot be taught through repeated practice of isolated skills. Students must understand enough of a passage to utilize all the intricately linked skills that together comprise comprehension. The owl question, for example, tests skills not learned from isolated reading practice but from processing information on the varying characteristics of animal species. (The correct answer, by the way, is the screech owl.)”
Data-driven instruction says teach the skill? Well, data-driven instruction is wrong. Reading is not a transferable skill with components that can be separated like an egg yolk from the egg white. Comprehension is a function of interwoven skill, prior knowledge and vocabulary. Expecting teachers to tease out a specific skill from the question Elden cites is like asking them to separate the yolk from a scrambled egg.
“Unfortunately, strict adherence to data-driven instruction can lead schools to push aside science and social studies to drill students on isolated reading benchmarks. Compare and contrast, for example, is covered year after year in creative lessons using Venn diagrams. The rersult is students who can produce Venn diagrams comparing cans of soda, and act out Venn diagrams with Hula–hoops, but are still lost a few paragraphs into a passage about owls. When they do poorly on reading assessments, we pull them again from subjects that give them content knowledge for more review of Venn diagrams. Many students learn to associate reading with failure and boredom.”
The expectation that teachers should use data in a way that belies what we know about reading is a prime example of what Rick Hess called The New Stupid – “a reflexive and unsophisticated reliance on a few simple metrics.”
“It’s impossible to teach kids to read well while denying them the knowledge they need to make sense of complex material,” Elden concludes “Following the data often forces teachers to do just that.”