The Inspector Will See You Now

by Robert Pondiscio
April 11th, 2012

If we look at more than just test scores to determine teacher effectiveness, shouldn’t we do the same for schools, asks Fordham’s Mike Petrilli.  The best accountability systems, he argues, “take various data points and turn them into user-friendly letter grades, easily understandable by educators, parents, and taxpayers alike.”  Petrilli wants to go one step further adding a human element to accountability in the form of “school inspectors” modeled on Great Britain’s inspectorate system.

Under Petrilli’s proposal, a group of inspectors would visit a school at least once per year. “They would mostly look for two things,” he writes.

  1. Evidence that the school is achieving important outcomes that may not be captured by the state accountability system. For example, the school’s administrators might show them test score data from a computer adaptive exam like NWEA’s that demonstrates progress for individual kids (especially those well above or below grade level) that isn’t picked up by the less-sensitive state test. Or perhaps a high school has compelling data about its graduates’ college matriculation and graduation rates that put its mediocre test scores in a different light.
  2. Indications that the school’s culture and instructional program are inculcating valuable attributes in their students. This is to guard against the “testing factory” phenomenon. Is the school offering a well-balanced curriculum (and extra-curriculars), or engaging in test-prep for weeks on end? Is it focused on teaching “non-cognitive” skills and attributes, such leadership, perseverance, and teamwork? Character traits like empathy, honesty, and courage?

Petrilli’s first point is deeper data that probably doesn’t require on-site inspections; the second is more interesting.  I’m all for a more nuanced view of school performance.  If you’ve been in a school lately and haven’t come away dispirited by the sheer volume of test-prep and frustrated by curriculum narrowing, you’re likely engaged in a form a denial or willful ignorance.  Anything that broadens the lens is a step in the right direction.

I certainly agree that you can tell a lot about a school by walking its halls and sitting in its classrooms.  The trouble is that the higher the stakes, the less likely you are to see—or to be allowed to see—the school as it actually is, warts and all.  I’m reminded of the spitting and polishing we used to do in my school when we were having a Superintendent’s walk-through or preparing for our “quality review.”  Suddenly fresh student work bloomed on every bulletin board.  Daily agendas were posted.  Aims and standards in child-friendly language were omnipresent on the blackboard.  Records and planbooks spruced up and made ready for review.  Amazing, engaging lessons were planned and delivered.  No boxed macaroni and cheese when company’s coming.   Dirty dishes went into the oven and dust bunnies were swept under the rug moments before guests arrive.

In short, it’s not hard to master the art of displaying “visible evidence” of teaching and learning, while the underlying practices remain disappointing.  If you think that test prep is a waste of time, try getting your lesson plans, student data, running records and myriad other bits of housekeeping presentation-ready for the Inspectorate.  Is this really what we want teachers to focus their energies on?

Some years ago, I proposed a system of random testing whereby the students to be tested, testing dates, grade levels and subject matter was a matter of chance.  The only way to perform well under such an accountability system would be to actually teach all of the students well in all subjects.  That still strikes me as the right impulse.  Any accountability measure with stakes attached to it will inevitably come to dominate classroom practice.  It’s simply human nature to want to put your best foot forward when your reputation or your job is on the line.  This is fairly obvious.  The most likely response to the accountability mechanism should be precisely the practice you want to see in classrooms.  Anything else misses the mark.

A good inspection system can add significant value.  A thoughtful review diligently considered can lead to constructive suggestions and improved outcomes.  It needn’t be a “gotcha” game.  But the same is true of pure test-driven accountability.  In theory, the best outcomes should come from a well-rounded curriculum, effectively implemented by well-trained teachers.  It just hasn’t seemed to work out that way in practice.

20 Comments »

  1. Yes, it is too bad “reformers” decided to kill Rhode Island’s inspection system so that they could “discover” this “innovation” a couple years later.

    Comment by Tom Hoffman — April 11, 2012 @ 7:37 pm

  2. Just a head’s up. On any given “reform” being pushed in US the UK and Australia are at least 5 years ahead so once you know the names in use you can track the controversy regarding the implementation there on something still being proposed here.

    Here’s what Theodore Dalrymple had to say a few months ago about what a charade these school inspections have been.

    http://www.city-journal.org/2012/eon0110td.html

    Robert-The intensive test prep is caused by refusing to either teach reading or solid content effectively and then cramming so that the effects of poor instructional practices will be less noticeable. Then the omnipresence of the cover-up technique gets used to push for even less objective accountability that would show the preferred non-academic emphasis.

    Especially true now that it’s getting so much harder to cheat. That’s why the establishment figures did not care if Hall and her staff were cheating. They viewed testing itself as illegitimate.

    Comment by StudentofHistory — April 12, 2012 @ 7:27 am

  3. Welcome Back SoH. You’re command of logic, facts and general erudition have been sorely missed.

    You’ll get no argument from me on the general principle of “effectively teach reading and content” and let the tests take care of themselves. That said, it’s pretty hard to pretend it’s sufficient for the entire K-12 spectrum. I’ll take that approach any day (and frequently counsel it) when you’re starting young. But it’s pretty hard to remediate 10-12 years of poor language proficiency, a small vocabulary, and a lack of background knowledge in middle school, for example, and show short-term results. Schools are under pressure to show results NOW, but language proficiency stubbornly resists NOW.

    There doesn’t seem to be a lot of room in the debate over accountability to recognize this, let alone allow for it.

    Comment by Robert Pondiscio — April 12, 2012 @ 10:17 am

  4. Student of History: It’s shocking to me that anyone who professes to value facts and evidence would link to Dalrymple’s outrageous and entirely unsubstantiated accusations. Nowhere in Dalrymple’s op ed does he offer the slightest bit of evidence to support his claim that “many” principals engage in elaborate deceptions when inspected. Indeed, since schools only receive one to two days notice of an inspection, it’s very difficult to believe that principals would have sufficient time to “draft well-regarded teachers temporarily” from other schools or to plan multi-day trips to amusement parks for badly behaved students. In fact, the House of Commons conducted an extensive and intensive study of “the role and performance of Ofsted” [the English inspectorate] last year and concluded that unannounced visits are unnecessary because the current short-notice system is working well. Please note that the members of Parliament interviewed many witnesses critical of Ofsted during their study. A very tiny number of educators will always find ways to cheat any accountability measure (just as some people will always find ways to cheat on their taxes). However, I see no evidence that “many” English principals engage in such elaborate (and, frankly, rather bald) deceptions.

    Comment by Craig Jerald — April 12, 2012 @ 11:09 am

  5. To Robert’s comment #3: Very, very true. Learning takes time and ES (especially, although MS does its share) wastes FAR too much of it, such that FAR too many kids enter MS without the foundation necessary for real HS-entry knowledge and skills by the end of 8th grade.

    I’ve noticed that CK schools that have opened in areas I’ve lived have all started with only a few of the youngest grades (k-1 or 2), and added a class each year because they explicitly stated that it was too difficult to make up for that lack of knowledge at later grades. They also typically didn’t allow transfer-ins at later grades, for the same reason, although I believe they had a testing program that allowed prepared kids (I was told often the home-schooled) to enter.

    BTW, not only language proficiency, but math/science proficiency and all of the other content area proficiencies have the same issue. I wish the ed world recognized and valued the concept of efficiency; don’t waste time. As far as I can see, it isn’t even on their radar screen.

    Comment by momof4 — April 12, 2012 @ 11:46 am

  6. Your point about short notice is well-taken, but it doesn’t change the larger problem: any inspection is going to be by definition (there’s only so much you can observe and so many hours in a day) somewhat cursory. I can’t be in every teacher’s room at every moment, so I’m going to look for “visible signs of learning.” When I taught in NYC, we knew “what they are looking for” so a tremendous premium was placed on having those things readily observable: bulletin boards with current student work; teacher-created charts and a “print rich environment;” students seated in groups; neat, organized and leveled classroom libraries; aim and standard on the board in child-friendly language, etc. ad nauseum. I had an AP who insisted, after visiting my room before a walk-through, that math manipulatives should always be in reach. She was not interested in my explanation that I was not teaching math at the time (I swear I’m not making this up). Why? “They want to see manipulatives within reach at all times.”

    Now, you could easily argue that I should have been doing these things anyway, and you might be right. But preparing for walk-throughs and visits prompted a state of high anxiety and busywork that was equally as wasteful of instructional time as endless test prep. The analogy I made earlier holds: if you’re having important guests for dinner, if you keep a clean house and serve well-balanced meals, then it shouldn’t be a problem, nor disrupt your routines. But for how many of us is this true?

    Comment by Robert Pondiscio — April 12, 2012 @ 12:04 pm

  7. School inspectors so administrators can pull off the same hoax on their superintendent as teachers do when the principal announces they’re coming in for an observation?

    How about zero notice. How about just showing up and witnessing the actual performance. This could actually add a tidge of credibility to the process.

    We’ve all seen the proverbial dog and pony show some teachers put on when the principal announces their upcoming evaluation. The teacher gets a new outfit, has her hair and nails done, stays after school late that afternoon and arrives early the next morning to prepare the room and the lesson for the “observation,” and presto, she comes across looking the second coming of Horace Mann. The problem; the very next day she shows up hung over, dressed in sweats and late for school, with no plans for anything that day.

    Again, anyone who has ever spent any time as a public school teacher has seen this charade from one of their colleagues, or worse, been guilty of it her/him(self).

    And no, Robert, getting your hair and nails done and purchasing a new dress will not earn you brownie points on your upcoming observation from Don Hirsch.

    Comment by Paul Hoss — April 12, 2012 @ 12:26 pm

  8. But it’s such a NICE dress…

    Comment by Robert Pondiscio — April 12, 2012 @ 2:06 pm

  9. Robert, You are describing an inspection system that is very flawed in design and implementation to begin with. Such a system would focus on meaningless isolated minutia such as “bulletin boards” that are easy to game. You seem to have a view of inspection visits as a shallow kind of “walk through” of hallways and peeking into classrooms. But it needn’t be that way. I’m talking about an inspection system where experienced professional inspectors spend a lot of time observing teaching, teachers, and students and evaluating instruction against rigorous defined standards and rubrics. It’s very hard to “fake” excellent instruction on short notice, especially for teachers who do not deliver such instruction on a regular basis. And students can tell inspectors if the lessons on inspection day are significantly different from the lessons they receive on other days.

    Comment by Craig Jerald — April 12, 2012 @ 2:49 pm

  10. Sounds great, Craig. But you’re not addressing my baseline concerns: even the most rigorous inspection over a couple of days will necessarily have to rely on a certain amount of observable “evidence.” And there will also be instructional time lost to spit and polish.

    To repeat, I think there is much to be said for a rigorous review that is driven not by compliance, but a spirit of improving teaching and learning. But I don’t see this as a conclusive form of “accountability.” Better than purely test-driven evaluation to be sure. But not perfect.

    Comment by Robert Pondiscio — April 12, 2012 @ 3:03 pm

  11. Hello Craig. I was not familiar with your work but I see you were the author of the relevant report on the English system suggesting it for the US.

    Here’s the link in case anyone else wants it:

    http://www.educationsector.org/publications/her-majestys-school-inspection-service

    I promise to read your report but Dr Dalrymple was I believe responding to a report in the Times according to that link. More importantly if anyone walks the walk in dealing with the results of bad educational policies and practices, it is a psychiatrist working with Englan’s prison population. That is the vein he writes from and I always find him worth listening to for that reason on education particularly.

    You know my childrens’ school just had a School Quality Review where the professional reviewers report was that the “teachers were teaching and the children were learning and that had to stop”. That quote was not all that shocking to me as I have been studying and documenting what is really going on in education all over the world. The reviewer was already on my radar screen for insisting a very inexperienced principal should be the “instructional leader of the school” over far more experienced faculty.

    Not to be mean Craig but it is really difficult for me not to see this school inspection push as a means of bringing recalcitrant principals into compliance with the nonacademic and leveling purposes that are really at the heart of the Common Core implementation.

    Just like the teacher classroom evals that are suddenly the rage, this seems to go back to the ways in which the 1960s attempts to change American education went awry. The school and classroom levels where principals and individual teachers continued to want to teach as much content as possible to the best of their ability.

    Those reforms as well as the 1990s attempt led to adaptation, not adoption with fidelity. This time the bureaucrats want no deviance allowed.

    Are you really mad about Dalrymple’s articles laying the blame for those awful London riots at the foot of what Blair and Barber’s reforms did to English education? I have those articles somewhere.

    Thank you for your kind words Robert. I was determined to figure out what was really going on with Common Core. It turned into a much bigger story than education. Both fascinating and tragic. Sometimes truth really is both stranger and more compelling than fiction.

    Off to read Craig’s report.

    Craig-if there is other documentation you think I should read, link it please.

    Comment by StudentofHistory — April 12, 2012 @ 3:07 pm

  12. Robert, I do take your concerns very seriously and think they are valid. I just don’t think they are insurmountable. It might be my natural optimism, but I think careful design and implementation of an inspection system can intentionally mitigate such risks. Let me be clear: No matter how the inspection system is designed, some principals and teachers WILL try to game it just as they will try to game any accountability measure including standardized tests. But if potential gameability disqualifies any potential accountability measure, we won’t be left with any measures. Bottom line: I believe that inspection systems can be designed to discourage but not eliminate gaming, and I believe that some amount of gaming might even be a small price to pay in order to enhance our current test-based accountability systems in three important ways:

    1. Judge schools on a broader range of evidence without losing sight of the fundamental importance of student achievement, including standardized test scores.

    2. Leverage expert judgment rather than relying solely on spreadsheet formulas, yet still ensure sufficient safeguards against inconsistent or inflated ratings.

    3. Achieve a better balance between rigorous evaluative ratings and better diagnostic feedback to help schools improve.

    Inspections might not be the only way, or even the best possible way, a state might pursue such goals for improving its accountability system. But I’m not aware of alternatives that would accomplish those goals better without introducing measures that also are gameable to some extent.

    Comment by Craig Jerald — April 12, 2012 @ 3:47 pm

  13. Student of History: Thank you for taking time to read the report. I hope you find it valuable, and I am eager to hear your feedback.

    Regarding the Times Education Supplement (TES) article that Dalrymple cited: The article did set off a flurry of outraged coverage in the tabloid press. However, soon after its publication, readers and others pointed out that the evidence in the article was untruly anecdotal and journalistically unsubstantiated. In fact, the evidence of cheating and gaming came from commenters in TES’s online forums! Imagine if the New York Times published a major “expose” of cheating on state tests based only on anecdotes supplied by commenters on its website. Not surprisingly, the original article disappeared entirely from the TES website shortly after its publication, and it remains unavailable. If you can find a link to it, I’d love to see it.

    Comment by Craig Jerald — April 12, 2012 @ 4:15 pm

  14. SOH: I hope you’re going to share your findings on Common Core with all of us. Don’t tease!

    Comment by James O'Keeffe — April 12, 2012 @ 5:58 pm

  15. Interesting thread. GreatSchools would like to provide insight to parents into the breadth and depth of curriculum and programs at schools, including ways that schools are helping students develop concognitive traits like empathy and persistence. A few thoughts:

    1. Might we be able to crowd-source these insights from parents students and teachers? This is tricky buisness because unlike say restaurants, parents, students and teachers usually are not familier with many schools and therefore crowd-sourcing might say more about the expectations of the crowd than the reality in the school. But this is interesting to us at GreatSchools because we’re reaching 40-50% of families and teachers in the country.

    2. Might parents and community members be trained and enlisted to do a lightweight kind of school inspection? Unannounced. They look for certain critial things and report back. It becomes a community engagement thing as well as a way to scale a process that can provide some insight into schools beyond test scores. I think this would be difficult but I’d love to hear from anyone who thinks this might be a good idea.

    Bill Jackson
    CEO
    GreatSchools

    Comment by Bill Jackson — April 12, 2012 @ 9:28 pm

  16. If I need to know my weight to evaluate my health, telling me the volume of blood I have matters not. We measure all kinds of information in schools, but are we measuring what we need to know? Do we need to know specific facts, ways of working ie a reference paper or powerpoint. Do we need to just know how to do the research (I hear this idea from a lot of parents and educators) would suspect that if you lined up 50 parents and asked them about the breadth and depth of a curriculum you would get at least 25 answers. Never mind political and social agendas. Think about the Tennessee law that was signed that from my understanding would allow a teacher to discuss Creationism.

    http://www.reuters.com/article/2012/04/13/us-usa-education-tennessee-idUSBRE83C0JR20120413

    Does this expand/deepen the curriculum? Not in my book but would to many parents. We still have the problem that we are building a rocket and some people just want to go to space, others to the moon and a few thought it was actually a submarine. None of these methods of measurement get us where we need to be if we don’t agree on what should be measured and what that evidence demonstrates.

    Comment by DC Parent — April 13, 2012 @ 12:06 pm

  17. Craig-

    I told you I would look at your report and I just finished it. I m afraid I stand by my original statement that these inspections are intended to change school practices and push the nonacademic focus. I found that focus on “pupils’ spiritual, moral, social, and cultural development” interesting. Partly because I know it is consistent with the School Climate Index that got mandated as part of Georgia’s NCLB waiver. Georgia is so willing to pilot all these affective elements parents there may soon be calling it the Gulag of Georgia.

    I also found the info on page 3 about Cambridge Ed to be interesting. The Quality Review Report with the quote on the need to stop the teachers from teaching is from Cambridge. Your report indicates my childrens’ schools are getting the English style inspections and that these inspections are not yet widespread.

    How ironic is that? Now what can I do with cutting edge info?

    Comment by StudentofHistory — April 15, 2012 @ 6:23 pm

  18. “In fact, the evidence of cheating and gaming came from commenters in TES’s online forums! Imagine if the New York Times published a major “expose” of cheating on state tests based only on anecdotes supplied by commenters on its website.”

    The TES forums are populated by teachers. Obviously, the experiences related by teachers are anecdotal, but to dismiss everything teachers say they experience as irrelevant to debate is about as absurd and undemocratic as it gets. Who knows better what happens in schools than teachers? We see this stuff happening, isn’t that enough? Telling us that we should ignore what we see and listen to some expert as the evidence of our own eyes is not good enough might sound objective, but it’s actually just a way to exclude teachers from the debate.

    Comment by Andrew Old — April 17, 2012 @ 1:22 am

  19. [...] Hyslop of Education Sector and Robert Pondiscio of Core Knowledge each responded to Petrilli’s proposal with some degree of skepticism. While [...]

    Pingback by Addressing Concerns about School Inspections — April 17, 2012 @ 11:53 am

  20. I see Craig has taken this discussion on inspections over to The Quick and the Ed. Since this discussion is not going stale I will raise an additional point here that I found troubling in that Education Sector report “On Her Majesty’s School Inpection Service”. In addition to my troubling experiences with Cambridge Ed which I have already described above, Craig’s report seeks an additional role for the regional accreditation bodies to conduct these inspections.

    First when 4 of the 6 regionals are now owned by a common body, AdvancEd, which prepares the accreditation standards for all. It is at best sloppy to make references to them as if they remained independent bodies.Secondly, I tracked down the Rothstein etal report “From Accreditation to Accountability” and read it because I find the very idea of giving AdvancEd even more authority or access to taxpayer coffers so alarming. Rothstein’s paper makes it clear he envisions these inspections as a way to gain access to states and districts not yet bound to mandatory accreditation.

    I think an objective analysis of what the accreditors have been and are now imposing as requirements for accreditation would show that they have become major drivers for the push in both K-12 and higher ed away from the transmission of knowledge. The standards show that the accreditation process has become a back door means of mandating social policies that the govt is constitutionally and politically (from a PR perspective) prevented from enacting directly.

    Throughout education including teachers ed program the accreditation process is a prime driver of bad policies. It is less visible but gets the policies and practices in place regardless of what the public rightfully assumes it is getting off for all those tax dollars being sucked currently into the ed vortex.

    Comment by StudentofHistory — April 17, 2012 @ 3:19 pm

RSS feed for comments on this post. TrackBack URL

Leave a comment

While the Core Knowledge Foundation wants to hear from readers of this blog, it reserves the right to not post comments online and to edit them for content and appropriateness.