Gone are the days when educators dismissed data as having only a limited utility for improving schools and school systems. What’s taken its place, argues Rick Hess, is “The New Stupid” — where data-based decision making and research-based practice “stand in for careful thought, serve as dressed-up rationales for the same old fads, or [are] used to justify incoherent proposals.”
In an article in Education Leadership, Hess describes first encountering the tendency to “energetically misuse data” during a presentation to a group of aspiring superintendents.
The group had recently read a research brief high-lighting the effect of teachers on student achievement as well as the inequitable distribution of teachers within districts, with higher-income, higher-performing schools getting the pick of the litter. The aspirants were fired up and ready to put this knowledge to use. To a roomful of nods, one declared, “Day one, we’re going to start identifying those high value-added teachers and moving them to the schools that aren’t making AYP.”
Now, although I was generally sympathetic to the premise, the certainty of the stance provoked me to ask a series of questions: Can we be confident that teachers who are effective in their current classrooms would be equally effective elsewhere? What effect would shifting teachers to different schools have on the likelihood that teachers would remain in the district? Are the measures in question good proxies for teacher quality? What steps might either encourage teachers to accept reassignment or improve recruiting for underserved schools?
My concern was not that the would-be superintendents lacked firm answers to these questions,” Hess recalls. “It was that they seemingly regarded such questions as distractions.”
The key is not to retreat from data, Hess counsels, ”but to truly embrace the data by asking hard questions, considering organizational realities, and contemplating unintended consequences. Absent sensible restraint, it is not difficult to envision a raft of poor judgments governing staffing, operations, and instruction—all in the name of ‘data-driven decision making.’”
This is smart, even heroic stuff.
The idea that “data-based decision making and research-based practice ’stand in for careful thought, serve as dressed-up rationales for the same old fads, or [are] used to justify incoherent proposals’” is misleading.
That’s not data-based decision making or research-based practice to begin with. Data-based decision making works. It can’t not work, it’s simply not possible.
What fails is the implementation: we have lots of educators and administrators who have finally accepted the idea of data-based decision making, but they’re going at it with no training after years and years of going with their gut. Of course it’s going to fail, it’s like they’re trying to play Flight of the Bumblebee the first time they’ve ever touched a string instrument. What’s worse is that it always seems to be reported as (or at the very least, phrased as though there is) a flaw in the instrument, when the flaw is actually in the performers.
It seems to me that Education Leaders pointed to in Mr. Hess’s article fail to put into practice what they preach. That would of course be the holy grail of education, “critical thinking.” I apologize for the cynicism, but I believe it is well earned.
Let’s not forget that the data teachers and leaders receive are not always ironclad–and often difficult to adapt to their purposes. The understandable pressure to base decisions on data, coupled with the paucity of good, usable data in some schools or districts, can cause problems–for the stupid and the intelligent alike.
Better assessments, better data systems, and better professional development on how to use the data once they become available can help inoculate leaders against data-based stupidity, new or old.
Sometimes I wish I had faith enough to say it’s simply not possible for data-based decision making not to work. And I wish I thought that failures rested even primarily with implementation and not with the research design which may be to say with that “holy grail of education, critical thinking.”
Great great story. A generation ago, or if those superintendents had worked themselves up the ranks, they would have known that a) you can’t move teachers around like chess pieces and b) it wouldn’t work. But the newcomers did not seek out institutional history.
I’m optimistic though, and Katherine Boo’s story on Micheal Bennet in Denver explains why. Boo wrote, “Other ambitious superintendents admit privately that radical reform has cllateral costs, and that students like Norberto (a student in a school closed by Bennet.” So
in comparison to “pliable second graders, teen-agers are a poor investment.” If the superintendents would just say that out loud, we could have a constructive discussion. And if we’re honest about the difficult in saving older teens, we can perhaps invest more in the early years in a more holistic manner. And if we’re honest about the continuing failure of neighborhood secondary schools, maybe we can have a more honest discussion of whether NCLB has been producing more elementary students who can decode but not comprehend, and that we need to be more balanced in both our curriculum and of non-curriculum efforts.
I have qualms about Duncan, for instance from what I hear about the excessive test prep etc., but I’m hopeful for the same reason that Boo made me hopeful about Denver. Bennet took the time to get to know many of the 558 students he tried to personally help, and presumably he learned when he mostly failed. Duncan plays ball with people and he’s mourned with people at funerals and crime scenes. He must have learned that education is a people business. Hopefully, it made him more modest.
Thinking critically about data is not a skill that is easily learned in one or two professional development session.
For all its power, data has the ability to mislead as well as guide. It’s not always complete, and it’s not always without hidden and unintentional biases. This is the case in many other fields besides education, even ones where data is obtained with more experimental controls (think hormone therapy to treat menopause…).
Data-based decision making will only work if the people making the decisions have learned to think carefully and critically about the data they are using.
My husband works (at least of today) in the financial services industry. They’ve had a lot of very bright individuals doing “data-driven decision making” in recent years and you can see for yourself the results :-0
Mr. (Dr.?) Hess is absolutely right that judgment and thinking through possible unintended consequences are crucial.
Example of the New Dumb:
Dumb school leader in traditional high-pov school sees successful No Excuses school. Anti-charter forces repeat mantra that kids in those schools learn b/c they’re “taught to the test.” Dumb leader listens and actually believes it.
So dumb leader then does rote test prep that accomplishes nothing — activities that do NOT happen in the No Excuses schools. Kids do not learn in Dumb leader’s school — plus good teachers there get frustrated. Per Dave’s comment above, they blame tests and “data.”
Mike G.
ROFL
I actually saw this thinking in action, back in the 80’s, in Cleveland, OH. Since the parochial schools had high academic achievement and few discipline problems, the district decided to fill openings one year with teachers recruited from the Catholic schools. Great idea, right?
Wrong. About 1/2 quit before the end of the year, and many of those that stayed were basket cases. They came in with confidence and a little bit of arrogance “we’ll show these city teachers how to teach!”
After less than a week, I started hearing the horror stories – children who didn’t respond to firm reminders to sit down and be quiet! Children who answered correction with “F— you!” Fights, disruptions, and chaos.
The regular teachers had trouble keeping from delivering a well-deserved “told you so!”
The fact is, the skill set that works for a pliable, academically ready child may not be the same as that used by the inner-city vet. The methods in the repetoire of the more “successful” teacher in high-performing school often don’t work in the “needier” section of town.
For example, city kids don’t “get” hints on how to behave – “Will you sit down?” is seen as a choice (to which the answer is often “no”), rather than a last-chance to avoid the boom being lowered. A direct command “you need to sit down” followed by a “thank you” AFTER the student complies is far more effective.