There used to be a different inspection office, called EQA. They inspected districts.

They just disbanded EQA.

Districts didn’t like critical reports and complained to the Governor.

In both cases the immediate purpose of the report is to justify the renewal of the school’s charter. In the second report, the less favorable one, I was struck by the four requirements that they school had to meet for its charter renewal. The first requirement for renewal was that the school had to come up with a report about something. Check. Done. Accomplished. The second requirement was that the school had to come up with a report about something else. Check. Done. Accomplished. The third requirement was that the school had to come up with yet another report on yet something else. Again, check. Done. Accomplished. They’re batting a thousand so far. The fourth requirement was that their students must make adequate yearly progress in NCLB terms. On this they came up a little short.

Some people would say this is all reasonable and sensible. Others, like me would be much more cynical. It appears to me that only one of those requirements is substantial. The other three, in my humble opinion, can be fairly described as jumping through hoops. They assemble some rhetoric saying that they have very good intentions. Is the world a better place now?

All this reinforces my perception that subjecting schools to regular inspections is nothing to get excited about. I stated in my last post that I believe valuable information could be obtained by spending time in a school and trying to find out what’s going on. But that seems much more like a possibility than a realistic expectation.

It seems to me that we don’t really have much agreement about what constitutes good or bad teaching, or what constitutes a good or bad school. We have some agreement, but that seems mainly intuitive. We think we can recognize what’s good or bad when we see it. Nor do we have much agreement about the means to accomplish what we intuitively agree on. So does this give us an adequate basis for making a system of school inspections beneficial, worth the cost?

It is suggested that such inspectors need training. Where will this training come from? Ed school? Where else? Is this good?

The last year I taught in a public school we had a school inspection. I was not much involved, and didn’t give it much thought. I got a copy of the report a month or so later. I read it, but without much interest. Life in the school went on as before. Nothing changed. I suppose no one expected anything to change.

More recently I was working in a factory when the factory had an inspection that was a part of some process for obtaining some desired certification. Again it was all very peripheral to me. It came and went. Nothing changed, of course.

When I was in the Army we had occasional inspections of our unit. They were always unpleasant, but of course there was never any change in anything.

Where will educational improvement come from? School inspections? Well, maybe. Apparently they have them in Massachusetts, and from what I read Massachusetts does pretty well in its education. But correlation, we must constantly remind ourselves, is not causation.

So what is the cause of Massachusetts’ success? I’d really like to know.

http://www.doe.mass.edu/charter/reports/inspect/0484.pdf

This is an example of a negative school review.

http://www.doe.mass.edu/charter/reports/2007/ncces/sitevisit.pdf

* * *

If you feel like reading it, simply search for the word “observe” and you’ll find what the inspectors reported first hand.

For example: “Site visitors observed that the quality of instruction delivered during the essential skills blocks was uneven, with teachers ignoring some off task behavior and sometimes focusing their attention on a small group of engaged students. The tone in some classes was observed to be friendly, casual, and lacking in focus or urgency.”

Still, my sense of the BBA group is that they’ve acknowledged the complexities of the approach they promote. Chris Cross said quite openly that the development of an inspectorate will take time. At least other countries offer some models.

Stuart–Fair enough about Jacob and Lefgren, who find that value added measures are more effective than principals in predicting future student achievement. Yet they also suggest that principals might be able to identify “strategic behavior on the part of teachers to improve test scores without increasing actual knowledge” (i.e. mindless test prep) because they “observe inputs as well as outputs.” A well-designed inspectorate system could have similar effects. And don’t forget that inspections would be coupled with other forms of data, including state assessment data.

Yes, I know what a lesson plan is back in ed school. Is that what we’re talking about here? But the lesson plans we learn to make in ed school are just a matter of jumping through hoops to get through the course. Everyone knows that. Don’t they? Haven’t we known that for about a hundred years now? Has anything changed?

Teachers plan. Good teachers obviously make good plans. But would anyone actually claim that a good plan by a teacher would be judged a good plan by ed school standards?

Yes, I know I’m a cynic. Don’t I have good reason? One might ask why I’m talking about ed school. Well, who is going to train and certify those inspectors?

Good teachers plan. They always have and they always will. But that doesn’t mean they write down those plans, or that if they do that writing will be little more than scribbled notes to themselves, or that those scribbled notes will have any meaning to anyone else.

I teach in college, lower level math. That is certainly different in some ways than teaching K-12, but in other ways it is very much the same. I struggle everyday to explain math in ways that students can understand, so that they can do problems and pass tests. I struggle everyday to make assignments that will be just what students need to understand. I struggle to get those homework assignments graded and back to the students the next class period. I then I struggle to figure out how to do it better next semester. I have some success, and I have some failure. But I plan. I plan every semester and I plan everyday. I spend considerably more time in planning than I do in front of a class.

But if an inspector came by and wanted to see my “lesson plan” for tomorrow’s algebra class, what I would show him or her is my assignment sheet, a list of problems. That’s it! That’s all I write down. That’s all I plan to write down. That’s all I have time to write down. That’s all I should write down. I would probably also show the inspector a bit of attitude, an attitude that would not be in my best interests. I have ideas about teaching and learning. I try everyday to refine those ideas, to profit from my past experience, to do things a little more effectively next semester.

Yes, I am capable of writing down those ideas. I have a website full of that writing. But to make a “lesson plan” as we learned in ed school, with goals, objectives, methods, etc, is not a good use of my time. Jumping through hoops seldom is.

A well designed homework assignment, which generally means a well chosen list of problems, is crucial to good teaching. It’s just as important as a good explanation in class. But how is an inspector to look at a list of problems, from a book he has never seen before, in a subject he doesn’t know, and say, “Yup, you’ve got a pretty good plan there.”?

This is not to say that no valuable information could be obtained by spending time in a school and trying to find out what’s going on. But how it could be done to be more constructive than destructive needs a lot of thought.

I’m intrigued by the Wilkerson et al. (2000) paper suggesting that student ratings can be the best predictor of whether a teacher is successful at raising student achievement. See http://www.rise.hs.iastate.edu/uploads/reports/360%20wilkerson,manatt,rogers&maughan.pdf

BBA doesn’t, as far as I know, advocate the abolition of state tests, so a good inspection system could give us a much stronger sense of what’s happening behind the curtain. Test-prep factories would likely have a very difficult time suddenly creating sham lesson plans, assignments and portfolios that would belie their actual practice. The inspectorate reminds me of research by the Chicago Consortium earlier this decade: Consortium researchers reviewed the effects of school reforms on the level of assignments and student work, because they felt test scores alone weren’t complete measures of quality. The results of such research–or inspections–can inform robust conversations about improvement. That’s not necessarily the case with test scores alone.

As for the ability to train effective inspectors…. Research more recent that the studies you cite, Stuart, suggests that principals are actually effective and distinguishing the most effective teachers from the least effective teachers, though they’re less effective with teachers in the middle range. (See, for example, Jacob and Lefgren, 2007–http://econ.byu.edu/faculty/Lefgren/Assets/papers/principals.pdf). According to Jacob and Lefgren, principals are about as effective as value-added measures in identifying teacher performance–though that’s not necessarily a high bar to clear. Also, principals all too often haven’t been given the time or training to perform effective teacher evaluations.

There are models of effective evaluation programs that an inspectorate system could possibly draw on. As with any reform, poor or hasty implementation can do great damage. As always, the devil is in the details.

The good news is it would be easier to do random, unannounced inspections than random, unannounced testing. But somehow I don’t think that’s what BBA has in mind.

In a famous 1987 study that summarized previous research, Medley and Coker pointed out that ??“almost all educational personnel decisions are based on judgments which, according to the ?research, are only slightly more accurate than they would be if they were based on pure chance.”? ? ?And in a noted 1995 book, Kenneth Peterson claimed that “seventy years of research on principal ?ratings of teachers shows that they do not work well. Well-designed empirical studies depict ?principals as inaccurate raters both of individual teacher performance behaviors and of overall ?teacher merit.”?

What makes them think that these so-called “professional inspectors” would do any better?

* Medley, Donald M., & Coker, Homer. 1987. “The Accuracy of Principals’ Judgments of ?Teacher Performance.” Journal of Educational Research 80 no. 4 (March/April): 242-247. ?

* Peterson, Kenneth D. 1995. Teacher Evaluation: A Comprehensive Guide to New Directions and ?Practices. Thousand Oaks, CA: Corwin Press. ?