The New York Times is agog about a move, funded by the Gates Foundation, to “develop a better system for evaluating classroom instruction.” The project involves scrutinizing tens of thousands of hours of classroom teaching to find correlations between particular teaching practices and “student achievement” (higher test scores). “The effort has also become a large-scale field trial of using classroom video, to help teachers improve and to evaluate them remotely,” the Times notes.
“Video lasts,” Dr. McClellan [a director for the Educational Testing Service who is overseeing the process] said, creating possibilities for dialogue among teachers about improving classroom techniques. “Colleagues can watch your video and say, ‘Right here — where you did that — try this next time.’ So the teacher learns a new skill.”
That could happen. Then again, maybe not. A tool — any tool — is only as good as the person using it. Same thing for data, including videotape. It is not an intrinsic good; it is useful only insofar as it is used to diagnose perceptively and respond wisely. And that’s not a sure thing. Talk to a teacher in a struggling school and chances are good they’ve had this conversation, or a variation of it:
“Make sure you have math manipulatives out when the instructional supervisor visits on Thursday. And make sure your students are working in groups.”
“Why?”
“She likes to see group work and manipulatives for differentiation. The research shows it helps kids learn.”
“What research?”
“It doesn’t matter. She wants to see manipulatives and group work. Make sure that’s what she sees in your room.”
Classroom teachers are too often the last recipient in a long game of telephone. Sophisticated and subtle research, larded with variables, caveats and judgement calls, gets passed from the field to journals to trade publications to conferences and seminars to district officials to instructional supertindents to principals to APs and coaches until it arrives in the classroom teacher’s ear in the form of a bumper sticker–or worse: a civil service compliance item on a checklist. Something we do because “they” want to see it.
Over at Larry Ferlazzo’s blog, he describes a process of videotaping and professional development as “one of the most significant professional development experiences I’ve had.”
“Our school, led by principal Ted Appel, has begun having Kelly Young, an extraordinarily talented consultant on instructional strategies who we have been working with for years, videotape our lessons (I’ve written much about Kelly in this blog). He then meets with us to review an edited version of the tape, with us initially giving our own critique and reflections followed by his comments. This process is entirely outside of the official evaluation process, and is focused on helping teachers improve their craft.”
Clearly, videotape can be a powerful tool for improving teaching, but so can observations. The issue is whether they’re used, as Larry suggests, to help teachers improve, or just another “accountability” measure or evidence in a gotcha game. How soon before this happens:
“Mr. Pondiscio, we’ve reviewed the videotapes of your classroom for the last two weeks, and we noticed you’ve only used math manipulatives in twice in your last ten lessons. We’ve also seen three instances of whole class instruction, including asking students to read out loud, which as you know is an instructional practice we frown upon. I’m afraid we’re going to have to write you up and put a letter in your file.”
The lead in the Times’ piece points out that over 90% of teachers get top marks in evaluations. The clear implication is that there is something wrong with the tools used to evaluate teachers. The old saying about a sloppy worker blaming his tools is true. The dirty little secret about evaluations is that they are typically used to validate and reinforce the observer’s take on a teacher. If your principal or AP likes or values you, you get a good observation. If not, it’s a tool to irritate, harass and make you want to seek a job elsewhere.
So videotape teachers. Go nuts. Put an unblinking eye in every classroom. I’m not impressed with state-of-the-art data collection. In the absence of state-of-the-art data interpretation and dissemination–and an honest commitment to improvement–it doesn’t amount to much. If videotape is used to “improve practice” will the findings be delivered to teachers in a more timely, coherent and useful way than research findings tend to be currently disseminated? If used for evaluation, will it be handled differently than existing observations in which what is seen is what the observer wishes to see?
Color me skeptical.
[...] The Unblinking Eye « The Core Knowledge Blog. [...]
Pingback by The Unblinking Eye « The Core Knowledge Blog « Parents 4 democratic Schools — December 6, 2010 @ 12:10 pm
If skepticism regarding educational fads has a color, I suggest it be a fiery red. Let’s just keep on spending more and more money on these edu-toys and pretend we are actually accomplishing something. After the basics – student ability and motivation and teacher knowledge – the rest is just crap. but seemingly irresistable crap for far too many people. What’s a few more wasted millions when you consisder the wasted billions we’ve already spent on ssuch things.
Comment by Charles Fritsch — December 6, 2010 @ 12:56 pm
Here’s my thoughts teacher quality and training:
1. Ongoing training, feedback, and motivation is important to constant process improvement. The educational system fails miserably to provide this to teachers.
2. We should strive to provide the best supervision, motivation, and feedback that we can. It will be imperfect. Cynics will point to the imperfections as evidence of failure. It’s not, it’s just imperfections.
3. Every industry’s human resource management system is riddled with problems that makes it slightly arbitrary or unfair. So what? The point is that it’s the best we got.
4. The current HR system for teachers stinks and is ineffective. One of the problems is that principals get rid of teachers by passing them to someone else, not be getting them out of the system. And that principals ability to motivate financially is limited. Does anyone say that the current system works well? Exclusive reliance on test results, even with value-added effectiveness, is probably not enough. Evaluation of classroom technique is definitely part of the puzzle.
5. “Use manipulatives and small groups” as a hint of how to appear competent, if it nudges a teacher who lectures and uses only the board, in the right direction, is a good thing.
6. I’ve interviewed hundreds of people for sales jobs. It’s maybe odd that I can meet with someone for 30 minutes, ask some questions, listen to the answers, and then pick the competent good ones. Mostly, I get it right. Not everytime. I suspect that in a short viewing of teaching performance, we can mostly sort the masters from disasters. Not perfectly.
Comment by john edelson — December 6, 2010 @ 2:10 pm
While these videotapes have the potential to be misinterpreted, they also have the potenial to make quality teaching techniques and approaches more appreciated and pervasive thoughout the teaching community.
One of the comments that Stigler and Hiebert made after their analysis of the TIMSS video study was:
“Collecting national samples of teaching can serve two important purposes. First, it gives us solid information about the processes of teaching and learning inside U.S. classrooms, information that is crucial for developing sound education policies. Efforts to improve student learning succeed or fail inside the classroom, a fact that has too often been ignored by would-be reformers. Setting standards for content and performance is an important first step. But student learning will not be improved merely by setting standards and holding teachers accountable. We must study directly the processes that lead to learning in the classroom, for if we do not understand these processes we will have little chance of improving them. Most other professional and industrial fields have determined that improving the quality of the processes is the surest road to improving products, but we in education have yet to learn this lesson. The videotape study of classroom instruction allows us to refocus on teaching processes, with the aim of improving students’ learning.”
So I share your skepticism that these videos may not be used appropriately. But a database of teaching (the good, the bad and the ugly) may be what we need to focus ed reformers on the need for improvements in classroom instruction.
Comment by Erin Johnson — December 6, 2010 @ 3:03 pm
Of course they have the potential to make quality teaching techniques more appreciated, Erin. I’m merely bridling against the suggestion that this approach is manifestly different as an evaluation tool than an observation. A tool can be used for any purpose the person wielding it wishes. If we want to use this to improve teaching, we can. If we want to use it punitively or to reinforce our preconceived notions about what works and demand compliance we can do that too. There’s nothing inherently superior about videotaping teachers as a means of evaluation.
Comment by Robert Pondiscio — December 6, 2010 @ 3:13 pm
When there is videotape evidence of teaching (that can be used to appeal to a third party), how do you think that administrators will adjust their comments?
From the feedback that you have been given (faddish nonsense about how to teach e.g. manipulatives), it seems as if your concerns are that these videos will do nothing to improve the administrators feedback/evaluation. But, when there is hard evidence (as opposed to just a one-person evaluation), evaluations can be appealed and it is easier to demand evidence from the administrators regarding their basis for critiqueing the teaching techniques.
In one sense, a video log can be/should be used to hold the admin accountable for their comments/evaluations. They had better justify/support their evaluations when there is hard evidence to support or refute their comments.
Comment by Erin Johnson — December 6, 2010 @ 4:18 pm
Robert,
Skeptical or paranoid? You shouldn’t be either. Let’s count this one in the column where the glass is half full, shall we?
I loved the Japanese lesson studies Stigler and Hiebert chronicled in their book. What a great way to witness effective lessons and they have a data base where a teacher can actually access these better lessons.
Sure, it’s significantly different than what we have done historically in our schools but if it works, let’s give it a shot. And last time I checked Japanese students were cleaning our clocks in most everything related to academics.
I’m with Erin on this one (no surprise). Administrators would have to watch their p’s and q’s on whatever they said in a teacher’s evaluation because there would be unquestionable documentation on what actually went on in the lesson.
Comment by Paul Hoss — December 6, 2010 @ 6:43 pm
I’m not paranoid, and if I’m off-base on this, I’ll stand down. I’m merely not convinced that the answer to what ails us is “more” — more accountability, more data, more money, more anything.
Interesting point on the watching the watchers, as it were. Unintended consequences, anyone?
Comment by Robert Pondiscio — December 6, 2010 @ 7:53 pm
Robert,
You’re NOT paranoid and I don’t believe anyone needs to stand down. Videotaping has infinite potential especially if used to improve instruction but it can also be problematic if misused. Perhaps it could be helpful if the watchers knew they were being watched.
The only time I was ever videotaped was a real eye opener. I used the word “now” indefinitely throughout the lesson as a segue from one point in the lesson to the next. Wow! Talk about a picture (in this case a video) being worth a thousand words. A bit of a shockah initially but a helpful learning experience nonetheless.
Comment by Paul Hoss — December 6, 2010 @ 9:19 pm
Robert,
I think you’ve hit the nail right on its head….
Larry
Comment by Larry Ferlazzo — December 6, 2010 @ 11:09 pm
I’m with Robert. The NY Times story blows the classroom taping idea out of proportions.
I’m not saying taping is ineffective. Look, an observer can always sit in class and give his recommendations to the teacher afterwards. The trick with taping is not that it allows for more expert eyes, as the NY Times unwisely claims – but that it allows for self observation in addition to the opinion of others. The self is one’s harshest critic. Watching your own image helps you get closer to the idealized picture you may have of yourself.
Taping helps much with classroom technique and demeanor. Note however that it helps little with mastery of the required academics, our collective sour spot.
I think the taping technique works best when entry-level teachers are recorded and are shown the images in front of the whole group under the lead of an experienced teacher. But the number of entry-level teachers in one school is always small. And while longer-serving teachers would also find the exercise useful for themselves, the payoff for them is smaller, as their classroom behavior is harder to change and their image of themselves is already more self-aware.
Of course, the technology for mass taping has been available since VHS, so there’s no technical revolution waiting in the wings for this one.
But where the NY Times story goes entirely off rail, it is when it suggests that classroom taping should be continuous and should be made part of an ongoing review process, next to the test score value-added. The whole idea of continuous taped classroom records is positively creepy. Reviews would be highly subjective, and the few ‘master teachers’ actually qualified to review tapes would hardly have the time to do it thoroughly. Not to mention, the incentive for teachers in this new world would be primarily to make things look good on tape.
And then there’s the legal question: don’t teachers have a right to refuse to be taped? I think not just teachers, but students as well do. It is the right for self-image. What if some small incident is taped, and months later it is blown out of proportion? What if the teacher had mildly chastised a student, but the microphone mis-recorded the tone of the conversation? Do you remember Howard Dean’s ‘Yaa-hoo’ that sank his primary campaign eight years ago? It was a misconfigured microphone. The most devious effect of continuous classroom taping would be the risk-averse culture it would encourage.
And the NY Times claims that all these tapes will finally allow us to observe how high test scores correlate to teaching practices. This may be their most foolish claim yet. We know very well what are the effective teaching practices. And incidentally we already have the tapes to prove it – see the upthread discussion about the TIMSS tapes, or look at Alan Siegel’s paper: “Telling Lessons from the TIMSS Videotape: Remarkable Teaching Practices as Recorded from Eighth-Grade Mathematics Classes in Japan, Germany, and the U.S.”
http://www.cs.nyu.edu/faculty/siegel/ST11.pdf
It’s certainly not the case that the desirable teaching practices are not known, or that they are not already fundamented by solid research. The problem is that all these practices are either ignored or viewed as suspect, because they involve teacher directed instruction, student individual study and an early push for abstraction, and these methods don’t go along well with some of the romantic ideas proselytized by our education schools and misused in our schools.
Comment by andrei radulescu-banu — December 7, 2010 @ 12:38 am
When all is said and done, observations — whether in person or on video — tend to serve one of two purposes: to help the teacher improve or to monitor their performance. My sense is that there is much more interest in the latter than the former. And that’s not a formula for success.
Comment by Robert Pondiscio — December 7, 2010 @ 10:17 am
As a former teacher, I agree with Paul Hoss that watching videotapes of themselves teaching can be very eye-opening and helpful to teachers (especially new teachers) when it comes to improving their craft. When I student-taught years ago, we each watched a video of our performance and I remember that everyone was caught off-guard by certain behaviors and/or mannerisms that they weren’t conscious of performing (such as using “like” or “come on” or a similar phrase all the time).
However, I must concur with Robert Pondiscio’s suspicions that the wide-spread use of these videos by school administrators will be more for the purpose of “catching” the teacher behaving badly (not following “group work” protocols, etc.) than to help teachers improve on their performance.
I, for one, would have been very reluctant to have administrators repeatedly videotape my teaching because I think an administrator could use one sound-byte to ruin a new teacher’s career. If you were an office worker, would you want a video recorder sitting in your office, recording your every move: every time you blew your nose or made an offhand comment? I wouldn’t, and I’d bet most teachers wouldn’t either.
However, it would be fun if the videos could be used to prevent administrators from playing favorites with teachers for purposes of evaluations. That is, if there is video of two teachers, both of whom are doing a similar job, but the principal rates one as highly effective and the other as needing improvement, that might be useful evidence in proving administrator bias. I’m not sure how much that would help teachers though, because my understanding of union contracts is that teachers can’t grieve the substance of an evaluation, only the procedure by which it is undertaken. In addition, my understanding it that untenured teachers can be fired for any reason, because they don’t have at-will protection until they receive tenure.
Comment by Attorney DC — December 7, 2010 @ 10:51 am
A time or two in comments to educational blogs about various topics I have proposed that we ought to compile extensive video records of teachers doing their everyday jobs. However the scenario I had in mind varies in some important ways from what seems to be presented here.
I have always conceived of video recording as being for the purpose of educational research, not for the purpose of evaluating teachers. A twenty minute classroom visit or a twenty minute video clip would be essentially the same thing, but the video clip could easily leave out important factors that would be readily apparent to a visitor actually in the classroom. A collection of recordings of 200 hours of a teachers performance would be pretty much irrelevant. No one is going to watch more than a few minutes of it.
And I have always conceived that video recording would be under the control of the individual teacher, and only for the benefit of that teacher. By my view, then, if I were a beginning teacher I would choose to always have the camera on in my room, preferably three or four cameras. Everyday I would file away the video record, which probably means transferring the digital video file to an external hard drive. I would sometimes review part of the video record, but only for my own purpose. No one would have the right to demand to see any of it, though at the end of a month or so, or maybe once a semester, I might be expected to choose a ten minute clip that I believe shows me at my best.
As a new teacher I might want to ask an experienced colleague, “What do you do when such and such happens?” Under my scenario my colleague can reply, “Do you have a clip? I’m not sure just what you mean.” Since the cameras are always running I can say, “Sure. I’ll find the clip and show you what I’m talking about.” More likely I would say, “Sure, it’s right here on this SD card, 52 minutes into the clip.” I feel this could be tremendously helpful at times. And if the video record is entirely under my control I need not fear negative repercussions.
In my scenario it might not be the video clips of myself that would be most important. More important might be the simple fact that I, as a beginning teacher, would be encouraged to ask my colleagues to see what they do in the classroom. My colleagues would be under no obligation to show me recordings of their teaching, but surely experienced and confident teachers would love to be of help. As a teacher I have never felt that I had a really good idea of what my colleagues do in the classroom. Teachers talk, of course. Give them an opportunity and get them started and they’ll talk a lot. But our language is very imprecise. What does my colleague mean when she says “I stare the kid down!”? She claims she does it. She claims it works for her. What does she actually mean? I’d like to see a video clip.
What I have described so far is neither educational research nor educational evaluation.. It is simply a means that a teacher (any teacher, but especially a beginning teacher) can use for reflection and communication.
But I think the possibilities for research are huge. However the research that I envision is not of the count-something-and-do-stats type. The educational research I envision is of the look-closer-and-you’ll-see-more type. An extensive video library would be like the field notes of a naturalist or anthropologist. Research comes in when those field notes are analyzed critically, and hypotheses and evidence developed. Using a video record to assess the performance of a teacher would be like an anthropologist using his field notes to decide whether the culture he is studying is good or bad. No, the goal of an anthropologist is to understand a culture, not to judge it. The goal of a genuine educational researcher, in my humble opinion, should be to understand teaching and learning, not to judge a teacher by preconceived notions.
To be sure, that has to be done at times. We must sometimes judge teachers by preconceived notions. Most schools (but maybe not all, not the very small schools I taught in years ago as a beginning teacher) want some formal method of evaluating teachers. It’s inevitably part of the job of being an administrator. But it’s not educational research. It’s not even close.
I think the idea of identifying effective practices by watching videos is limited and misleading. My colleague says, “If a kid misbehaves, stare him down! That’s what I do.” Is that an effective practice? Real educational progress, in my opinion, would be when someone can look at a video clip of my colleague employing that practice and explain in terms that I understand why it works for her. Then I could understand why it’s a disaster for me, and what I should do instead.
I come by my thoughts and opinions based partly on my perspective that American education is not a disaster. Overall it works, and works pretty well. Most teachers, at least once they have a year or so of practice, do okay. Education is not broken. It doesn’t need to be “fixed”. Indignant calls to “fix” education imply that the solution is obvious. I don’t think there is any obvious solution. There is room for improvement, a lot of improvement perhaps. But it is not at all obvious from which direction improvement will come. And I think real improvement, however it comes, will involve supporting teachers, not shaking them up. It’s easy to see Robert’s caution about extensive video recording. It could certainly be used to shake up and threaten teachers.
I have an extensive video record of my teaching. Last fall, in what turned out to be my last year of teaching, I realized my digital camera will make movies. And I realized I’d like to have an extensive record of my teaching. I checked with the administration, was told there’s no problem if I’m only recording myself, not students, and so proceeded. For most of the year I just set my camera on myself at the beginning of each of my classes. I watched a few of the videos, and learned a little. At one point I edited out a three minute segment that illustrated a pedagogical question I thought might get some interest in among my colleagues. I didn’t elicit anything more than polite interest which soon vanished (which was a lesson for me in itself). I won’t claim any magic from video records, but I’m glad I recorded what I did. I’ll probably never look at those videos again, but if I want to, they’re there.
So I’m all for extensive video records being made of every teacher, but only in the way I have described. I share all of Robert’s apprehension about the Gates approach.
Comment by Brian Rude — December 9, 2010 @ 8:30 am
When there is videotape evidence of teaching (that can be used to appeal to a third party), how do you think that administrators will adjust their comments? From the feedback that you have been given (faddish nonsense about how to teach e.g. manipulatives), it seems as if your concerns are that these videos will do nothing to improve the administrators feedback/evaluation. But, when there is hard evidence (as opposed to just a one-person evaluation), evaluations can be appealed and it is easier to demand evidence from the administrators regarding their basis for critiqueing the teaching techniques. In one sense, a video log can be/should be used to hold the admin accountable for their comments/evaluations. They had better justify/support their evaluations when there is hard evidence to support or refute their comments.
Comment by Sharron Clemons — December 21, 2010 @ 4:20 pm