Run, don’t walk, over to the Washington Post to read Jay Mathews piece on the evaluation given to an AP History teacher under Washington, DC’s new IMPACT system for assessing teacher performance. Dan Goldfarb, a teacher at the Benjamin Banneker Academic High School has taken an extraordinary risk by giving a copy of his evaluation to Mathews, but in doing so, he has shed a lot of light on what sounds like a curious and capricious process.
Goldfarb was dinged in his evaluation for — among other things — two students passing notes in class and another (a straight-A student) who was not taking notes at all (poor student engagement). He also earned only two out of four points because, in the evaluator’s opinion, “there was little verifiable evidence apparent during the observation that Mr. Goldfarb works to instill the belief that students can succeed if they work hard.” Goldfarb responds: “Be a cheerleader and tell them that hard work is the key to success? Every five minutes or so? Are you serious? We are dealing with young adults, not small children.” Mathews is promising to follow up with evaluations from teachers who like the new system, but he seems sympathetic to Goldfarb’s criticism:
Overall, the evaluator gave the teacher only 2.3 out of a possible 4 points. Goldfarb got only 1 out of 4 points in one section for failing to post or say what the objective of the lesson was–to me unnecessary kid’s stuff for an AP class. He also got only 1 out of 4 points for not catering to multiple learning styles, even though some experts, like Willis D. Hawley of the University of Maryland, call learning style analysis “bunk.”
This is exactly the issue Dan Willingham raised over IMPACT a few weeks ago. It’s troubling, to put it mildly, that we’re now seeing teachers criticized for not catering to learning styles despite “utter lack of evidence to support it.”
At Harry Potter and the Urban School Nightmare, another teacher who supports Rhee’s efforts to get rid of bad teachers (who wouldn’t?) describes getting stellar marks he didn’t deserve.
I received a score of 3.8 (out of a perfect 4), which puts me in the “highly effective” category. Now, if I’d actually earned that score, I’d be pleased. But I didn’t. My lesson showed me to be effective, but not outstanding. So why did I get the score I got? Because my principal has decided that she likes me. Of course, this isn’t really a problem for me (except that I’m not really getting any feedback for improvement, I suppose). But it is a problem for the people she’s decided she doesn’t like. Some teachers at my school are unhappy with their scores, and for some I don’t really doubt that it’s because they’re not based in reality.
A commenter on Mathews’ blog nails it, noting “No teacher does all the things (all 25 or so of them) in 30 minutes. It is just not going to happen. I can do a dog-and-pony show for the master educator, or I can teach effectively and not hit all of the things that he or she wants.”
Keep your head down, Mr. Goldfarb.
I often didn’t take notes in class in high school, especially math class. I remembered the lesson better if I listened closely and thought about what the teacher was saying, without the distraction of writing things down.
Today if I did this I would be taking points away from a teacher. Awful.
Comment by Diana Senechal — November 23, 2009 @ 12:15 pm
Makes you long for the good ol days of value-added assessment…
Comment by GGW — November 23, 2009 @ 1:47 pm
Although I have heard it as a suggestion in EVERY professional development and grad class I have taken, I never write the objective or goal or standard clearly on the board. At this point I would almost refuse to do it just on principle.
Comment by Matt — November 23, 2009 @ 2:01 pm
If there were two students taking notes differently and another who wasn’t at all, doesn’t that indeed indicate that there are learning styles? People can have information (eg. Know when Columbus sailed to San Salvador) and process it differently(one thinks he “discovered America” and other thinks he didn’t at all). There are learning styles because people can view the world from different perspectives and interpret information differently based on what they learn on their own and in other places and though other people. No two kids will be alike. I’ve noticed that neither you nor Mr. Dan Willingham have ever provided studies that REFUTE the reality that kids have different learning styles. All you can do is argue against the fact ie. a logical fallacy.
Doesn’t the fact that some people think that the ratings they got they didn’t deserve and others think they did even though to someone else they didn’t reveal the inherent limitations in grading in the first place? Of course the situation with this teacher and others proves that people’s opinions are being made into something that they aren’t: objective. People’s opinions of the effectiveness of the lessons will vary on many different variables in the classroom and in any case the most important viewpoint and the only one that in my assessment that comes anywhere close to objectivity is the students’ themselves. I’ve never seen you, Mr. Willingham, or anyone else acknowledge that fact.
Comment by Anonymous — November 23, 2009 @ 2:37 pm
I was another non-note-taker. It bugged by A.P. History teacher, but I got a 5 on the exam, so I must have absorbed most of it.
In college I found I needed to take notes — the data rate was much higher.
Comment by Rachel — November 23, 2009 @ 2:43 pm
I think, Anonymous, that what you’re pointing out is precisely the point. A teacher who insists that students take notes while listening to a verbal lecture is not “differentiating instruction”; but then, if students do not take notes, the teacher is punished. It’s a no-win situation. The solution is to simply allow each teacher to direct the classroom environment as is appropriate to the teacher, the students, and the subject matter.
Comment by Miss Eyre — November 23, 2009 @ 3:16 pm
GGW states, “Makes you long for the good old days of value-added assessment.” Right on, baby. Right on.
I’ll teach my class the way I believe it should be taught. At the end of the semester you can test my kids. If they’ve performed poorly THEN you can criticize my methods, but if they’ve tested as well as I’m sure they will, stay out of my classroom and away from my kids. They’re doing just fine without all your subjective gobbledygook to judge them or me by, thank you.
“…some experts, like Willis D. Hawley of the University of Maryland, call learning style analysis “bunk.” Can’t say as I disagree with this Hawley character. It’s the one component of Carol Tomlinson’s differentiated instruction I believe she places too much emphasis on, at the expense of customizing/individualizing the PACE of instruction for each student in the classroom. I’m not saying there aren’t kids who learn better one way versus another (auditory, visual, etc.) because I believe there are. I’m simply stating Tomlinson places too much emphasis on this aspect of her philosophy. You can accommodate the different learning styles almost within the hidden structure of the operation of the classroom. It doesn’t need to be as formally acknowledged as she suggests.
Comment by Paul Hoss — November 23, 2009 @ 4:40 pm
Anonymous might be interested to know as well that there are no studies refuting the existence of gremlins, angels or the bogeyman, too.
I suggest for him or her a basic course in epistemology.
Comment by bill eccleston — November 23, 2009 @ 8:06 pm
Several years ago when I was teaching in California, I had the unfortunate experience of working for a principal who really disliked me from the start. When she evaluated me, she focused only on the negative things in the classroom. I remember a specific day when the principal and another observer both watched my class. You wouldn’t have known from reading their evaluations that it was the same class! For instance, if 19 of the 20 students were following directions, the other evaluator wrote “class on task and following instructions” but the principal wrote “student X by window ignoring directions.” Both observations were factually correct but they obviously convey very different impressions of teacher quality. Goes to show that “impartial evaluations” are largely a figment of the imagination.
Comment by Attorney DC — November 24, 2009 @ 9:25 am
To me, the most disappointing part about Mr. Goldfarb’s evaluation is the idea holding the whole thing up: that “good teaching” is ultimately quantifiable.
Good teachers tend to do X. Did Mr. Goldfarb do X? To a degree. 2 points of 4.
Good teachers tend to do Y. Did Mr. Goldfarb do Y? Yes! 4 points of 4.
…and so on.
It’s something about the whole evaluation system that always baffled me as a teacher. How, on one day, would I ever do all the good things “the literature” says I should do? Where is the latitude that should be allowed for the kind of task the class is working on that day? Or for how, knowing the class and my craft as well as I do, I’ve adjusted my delivery style to the roomful before me?
In the end, Mr. Goldfarb’s experience does not surprise me. It is perfectly in line with how schools are handling the accountability push. It is disappointing, though, especially in how the profession has responded to the data-uber-alles thinkers currently ruling school-improvement efforts. Administrative bodies–largely made up of former classroom educators but who have clearly lost touch with the more data-averse parts of the craft–may demand it, but where’s the teaching community’s pushback?
I’d like to thank Mr. Goldfarb for coming forward with how it’s being done in his school. Now my hope is that others will listen.
Comment by Eric Kalenze — November 24, 2009 @ 11:41 am
To Eric Kalenze: Well said!
Comment by Attorney DC — November 24, 2009 @ 12:46 pm
Sigh. I have to say that my evaluations when working in the insurance industry were more well thought out and well rounded than most of the evaluations from my classroom years. The evaluations that helped me to improve in the classroom came from an administrator who was also a very good teacher and now is doing a very good job leading an urban school. Not by coincidence, he “get’s it.”
I agree with Paul in many ways. A “drive by” evaluation of my classroom elicited remarks such as “should continue to check for understanding”. When I questioned this and explained that the lesson was going well and most students understood the lesson, allowing me to then focus my attention on a few individual students, my evaluator continued to argue the point. My question then became, “at what point do you stop checking for understanding?” There never was an answer to that. He had maybe 3 years of classroom experience.
It was my class, I had been with them every day for months and I knew them pretty well. Don’t bang the teacher over the head with the technique of the day if you are only going to be there for 50 min.
Comment by CaliTeacher — November 24, 2009 @ 12:47 pm
Thanks to anonymous for pointing out learning styles in a way that I had not thought of. It was not so long ago, perhaps a few years, that I first heard of learning styles. I thought the idea probably made a lot of sense, and would be beneficial in education. However it quickly became clear that the only “learning styles” anyone had in mind was visual, auditory, and kinesthetic. That didn’t seem very sensible or helpful. Those are sensory modalities. What do they have to do with learning styles? Should we bring the vestibular sense? Should we talk about olfactory learners?
I still think the concept of learning styles may eventually produce something useful to education. I just don’t think learning styles will ever be shown to have much to do with sensory modalities, except in cases where there is a special deficiency or limitation to one of those senses. But the idea of two students who take notes in substantially different ways and a third student who takes no notes, now that, it seems to me, is something worth investigating. I don’t know where it might lead, but it might lead somewhere. What are those students thinking? And what are they not thinking that perhaps they should be thinking?
I remain deeply suspicious of the idea that people learn in fundamentally different ways, but certainly one ought to keep an open mind.
And may I comment on Eric Kalenze’s comment. He says:
Good teachers tend to do X. Did Mr. Goldfarb do X? To a degree. 2 points of 4.
Good teachers tend to do Y. Did Mr. Goldfarb do Y? Yes! 4 points of 4.
…and so on.
Compare that to how we rate cooks. Okay, to be honest, I don’t rate cooks. But if I did perhaps I would compare what this cook does to what good cooks in general do.
Good cooks use salt. Cook Goldfarb uses salt. Sometimes. 2 points out of 4.
Good cooks use catsup. Cook Goldfarb uses catsup. Lots of it. 4 points out of 4.
. . . and so on.
But, as I say, I don’t rate cooks. I know nothing about cooking. Is there a better way to do it?
Comment by Brian Rude — November 24, 2009 @ 4:44 pm
Thanks, Brian! I love teaching-cooking comparisons, because they’re two crafts that are so similar in so many ways. (Also, somewhat conveniently, I happen to know a lot about both.)
At bottom, such a comparison shows just why putting teachers on black-and-white scales of effectiveness is so limited. Simply, a statement like “Good cooks use salt” does not apply in all cooking situations. Too much salt to a cake, for example, would ruin the final product. Similarly, adding salt to a dish with a lot of already-salty ingredients–just because cooking schools say “good cooks use salt”–will push that particular dish past pleasant.
Truly good cooks taste as they go and make necessary adjustments. They rely on their practice-trained palates and tastebud-visions of what the dish should taste like when complete. They can anticipate the effects of a little extra time on the stove, as well as what will result when they put a little extra or a little less heat beneath it. After a while, if they possess solid visions about their flavors and have good command of the techniques needed to achieve them, they can delight their guests without much need of a recipe.
Mr. Goldfarb, who started this whole discussion, sounds like he knows a thing or two about his saucepan–er, classroom. (And again, I’m just taking that from his words to Mathews. It may well not be the case. My personal experience and intuition, however, both tell me that his account is sadly accurate.) That someone keeps bugging him with the “supposed-to-do recipe” of effective teaching when, let’s be honest, no one seems to have come up with the perfect formula in nearly 150 years of public ed in America, is unfortunate. It’s the place we’re in right now, believing we can fix education by listening to the Jim Collinses of the private sector: make it more like a business (e.g., “Put your employees on a meeting pulse! Set metrics and let them inform all decisions!” et al) and your school will be a Factory of Dreams! Only time will tell if this is an effective approach, of course. Because I love kids and wish we could give them the best places of learning possible, I would love to be proven wrong…but I just don’t see it as the right answer.
Another small hole in your comparison: while you cannot imagine a better means of assessing cooks (teachers) for someone like yourself–who freely admits to not rating cooks or knowing anything about cooking–the people who rate teachers are supposed to be the institutional EXPERTS. Goldfarb’s observer was a learned “Master Educator”, for instance, and building principals and vice principals routinely handle this duty. Though I generally had great experience with this (I was blessed with incredible administrators when I taught), I know of many, many teachers who found themselves at the mercy of administrators with very little classroom experience who, predictably, applied only the “recipes” to the practice they observed.
Comment by Eric Kalenze — November 24, 2009 @ 6:20 pm
Brian,
It seems as though a good way to rate cooks is by tasting the food they’ve made. If the presentation is acceptable and it tastes good…
Comment by Paul Hoss — November 24, 2009 @ 6:23 pm
Cook A is given two pounds of kobe beef, farm-fresh produce, a well-stocked cupboard and a sous chef. Cook B is given two pounds of stew meat, a few cans of assorated vegetables and a can of evaporated milk. May the best chef win?
Comment by Robert Pondiscio — November 24, 2009 @ 6:54 pm
Now THAT is a dark way to look at it, Robert.
How about this: Cook B’s dish has a chance of challenging Cook A’s, but any kinds of time constraints would have to be thrown out.
Can education produce a reasonable, realistic analog?
Comment by Eric Kalenze — November 24, 2009 @ 7:02 pm
Not trying to be dark. Merely pointing out the difficulty–as many others have noted–of trying to come up with a fair and objective measurement apparatus. We may want a standard, objective criteria. But as Mick Jagger said, you can’t always get what you want.
Comment by Robert Pondiscio — November 24, 2009 @ 7:10 pm
I meant the cooking comparison as a joke, to ridicule the evaluation that Mr. Goldfarb was subject to, but maybe it wasn’t quite as transparent as I thought. Maybe I should have put in my thought about the cook who was whipping up a batch of strawberry ice cream but was rated low because she used neither salt nor catsup.
It’s true I know nothing about cooking, but I have long considered the parallel with teaching to be apt. In both we have ingredients, some basic and some exotic, that can be combined together in various ways. Some of those ways will be good and other ways will be not so good. Some ingredients are used in practically every dish, and other ingredients are to be used very sparingly. In both teaching and cooking (and perhaps other things) we ought to know something about the nature of those ingredients, and how they work together.
I extended this teaching-cooking parallel a little in one of my articles on my website. Here’s a link. http://www.brianrude.com/Tchap17.htm
Comment by Brian Rude — November 24, 2009 @ 7:11 pm
It’s been nagging at me all week why this form of evaluation bugs me, and the cooking analogies finally drove it home: this is how I used to be evaluated when I was 16 years old and working at a Taco Bell. We were told that the district manager had inspectors come in posing as customers. They were looking for certain things: Did we greet the customer with a smile? Did we suggestive sell a drink or side order? Did we thank the customer for coming, give him correct change and a receipt? There was a list of things we had to say and do every time, without fail, and we could be written up or fired if we didn’t do them all, every time without fail.
The goal (in business terms I was ignorant of at the time) was to insure a consistent customer experience. And that’s precisely what this feels like: trying to insure a consistent learning experience.
To state the obvious, teaching is not the same as working the drive-in window.
Comment by Robert Pondiscio — November 24, 2009 @ 11:11 pm