Value-Added: When Being Right Isn’t Enough

by Robert Pondiscio
August 17th, 2010

The Los Angeles teachers union may be correct to fight the publication of individual teachers’ value-added test scores.  But they’re on the wrong side of history, writes Dan Willingham, who has long made a compelling, fair and utterly dispassionate case (he has no skin in the fight) that using value-added measures to evaluate teacher quality is “not ready for prime time.”  He’s explained the problems in blog posts, and even in a YouTube video

Clearly, to no avail.  For those who have just arrived on the planet this morning, the Los Angeles Times over the weekend produced a blockbuster piece of reporting, based on years of test scores for 3rd through 5th grade teachers, enlisting a statistician to rate the effectiveness of individual teachers by name. The data, says the paper tells “which ones have the classroom magic that makes students learn and which ones annually let their students down.”  Blogging at the Washington Post’s Answer Sheet, Willingham responds:

“The writers of the Times article are either uninformed or disingenuous about the status of the value-added measures. They write ‘Though controversial among teachers and others, the method has been increasingly embraced by education leaders and policymakers across the country, including the Obama administration.’  The ‘others’ include most researchers looking into the matter.”

The L.A. teachers union is calling for a boycott of the paper.  Good luck with that, is Willingham response.  “When it comes to value-added measures, teachers and unions are right. The models aren’t reliable enough to evaluate individual teachers,” he observes. 

“But right now that doesn’t matter much. The mood today is that something has to be done about incompetent teachers. We’ve seen that mood in districts in New York City and Washington D.C. and now we’re seeing it in Los Angeles.  We’re also seeing it at the federal level. Education Secretary Arne Duncan said that the publishing of the individual teacher’s scores is just fine. The people who feel that something must be done are right. In most districts there is not a mechanism by which to ensure that incompetent teachers are not teaching.”

This is the time for the teacher’s unions to make teacher evaluation their top priority, Willingham concludes. “If they don’t, others will.”

Others have.

Conflicts of Interest

by Robert Pondiscio
August 3rd, 2010

Which is worse, asks A-Rus at This Week in Education:  cheating or plagiarism?  This after yesterday’s NY Times story on allegedly fungible definitions of plagiarism and an apparent vindication of Atlanta’s schools in the “Erase to the Top” scandal.

Just wondering:  Has the pressure on schools and teachers to measure up fundamentally changed the dynamic of cheating?  In a gentler age, cheating was how you put one over on the teacher.  Now, the teacher theoretically benefits from cheating as much as the student.  Maybe even more.

David Steiner Gets It

by Robert Pondiscio
July 14th, 2010

Keep an eye on New York State education commissioner David Steiner, who is gearing up to implement a long overdue reform: establishing a link between test scores and college readiness. 

Harvard’s Daniel Koretz, at Steiner’s urging, has been looking at the correlation between New York’s eighth-grade test scores and high school Regents exam scores. Notes the Buffalo News:  ”The conclusion: Students in New York State are moving through elementary, middle and high school with test scores they believe to be adequate, but once they get to college, they find they are not prepared.”  That’s not a complete shock given the boxcar numbers of college freshman who need remediation once they arrive on campus.  But the New York Post’s Yoav Gonen points out what will surely be the most repeated fact from Koretz’s forthcoming study: eighth-graders who score a 3 out of 4 on state math and reading tests have just a 52 percent chance of graduating high school, even though they’ve been told they’re on track.

Let that rattle around inside your head for a moment:  A child who is deemed proficient in 8th grade has a chance only slightly better than a coin toss of graduating high school just four years later.   “We’ve been calling that ‘proficient,’ ” state Board of Regents Chancellor Merryl Tisch told The Post’s editorial board. “We were giving out misleading information.”

Gee, ya think?

The study is to be released Monday, but anyone who has taught in New York in the last several years can’t be surprised.  For years, I saw 5th graders come into my Bronx classroom who were ostensibly on grade level yet demonstrated little command of basic arithmetic. That was plenty persuasive that all that glitters isn’t gold.

Steiner’s insistence that test scores should actually mean something is clearly going to rattle some cages, and prompt a long hard look at where school districts in New York have made real gains and where they haven’t.  Buffalo’s school superintendent blasted Steiner and his deputy John King last week for focusing on more rigorous tests.  ”I think they’re two people who don’t know what they’re doing,” James A. Williams told the Buffalo News. “A more rigorous test is not going to improve student achievement. It’s not going to improve the graduation rate. I think it’s ridiculous.”

I don’t follow Williams’ complaint.   By my read, Steiner isn’t talking about testing our way to proficiency.  He’s talking about how test scores should be indicative of real-world proficiency.  As I’ve argued in this space before, if we’re going to insist on viewing everything in education through the prism of test scores, those scores have to be meaningful and indicative of real-world proficiency.  Steiner, King and Tisch deserve all the credit in the world for taking this on.

Joint and Several Accountability

by Robert Pondiscio
May 19th, 2010

On a Wednesday afternoon in May, Mr. Jones is walking down a city sidewalk, minding his own business.  At the end of the block, Mr. Smith, an uninsured motorist, sees the light turn yellow as he approaches an intersection.  To beat the light he floors it, just as an indigent man named Mr. Baker steps into the crosswalk. Smith swerves, hits a pothole and loses control, careening onto the sidewalk where he hits Mr. Jones, who is seriously injured.

Who does Mr. Jones sue?   The driver who hit him?  The jaywalker who caused Mr. Smith to swerve?  The answer is probably both – and the city, since they are responsible for maintaining the street and the sidewalk.   At trial, Mr. Baker is held 50% responsible for Mr. Jones’ injuries.  He was crossing the street ignoring the ”Don’t Walk” signal.  Mr. Smith is 45% responsible.  He had the right of way, but was reckless in speeding up as he approached the intersection.  The city is deemed 5% responsible. Jones’s lawyer persuaded the jury that if it weren’t for that pothole, Mr. Smith would not have lost control of his vehicle.

The jury awards Mr. Jones $30 million.  Mr. Smith has no insurance and few recoverable assets.  Mr. Baker has no property whatsoever.  The city can pay, but they’re only responsible for 5% of the damages, right?

Wrong.

Under the principal of “joint and several liability,” some form of which is allowed in nearly every U.S. state, the city is responsible for all the damages. Unfair? Perhaps, but Mr. Jones’ injuries were not his fault, so he who can pay does pay so that Jones can be made whole.

In education, we are moving ever closer to a similar system.  Call it joint and several accountability.  A child may fail in school for any number of reasons—uninvolved parents, poor attendance, lack of motivation, poverty, hunger, or an unstable home life.  That child is surely as damaged as Mr. Jones.   However, since we have no means to hold parents, peers or poverty accountable, the full weight of accountability falls on the teacher.   Like the deep-pocketed municipality in an accident, she is the only one within reach, even if she is only partially responsible.  

“The U.S. is one of the few countries in the world that places virtually all the burden for learning on the shoulders of teachers,” notes veteran teacher Walt Gardner, at EdWeek’s Reality Check blog. “This notion is alien to teachers in most countries that are our competitors in the new global economy. Yet it gets scant attention from reformers,” he notes. 

Indeed, even if you believe that accountability as a “theory of action” can address the problem of ineffective teachers, it doesn’t solve  the larger imbalance of responsibility for learning.  It will no more guarantee success for all students that the threat of punitive damages will guarantee accident-free streets.  Personal accountability and intrinsic motivation will always matter more.  After all bad teachers are identified and fired, Gardner concludes, “the problem of balancing responsibility for learning won’t go away. It’s largely an American phenomenon.”

“We Must Break Free of the NCLB Mindset”

by Robert Pondiscio
April 5th, 2010

“We wasted eight years with the ‘measure and punish’ strategy of No Child Left Behind. Let’s not waste the next eight years,” writes Diane Ravitch in a  Washington Post op-ed piece.  Criticism among ed reform standard bearers of Ravitch’s book, The Death and Life of the Great American School System: How Testing and Choice Are Undermining Education, has largely centered on her lack of alternatives to accountability and school choice (as if there’s nothing to be gained from diagnosing the shortcomings of current preferred reform strategies).  Her Post piece lays out specific ideas to support her main thesis, that school improvement is a function of curriculum and instruction, not structural reforms.

To begin with, let’s agree that a good education encompasses far more than just basic skills. A good education involves learning history, geography, civics, the arts, science, literature and foreign language. Schools should be expected to teach these subjects even if students are not tested on them.

Everyone agrees that good education requires good teachers. To get good teachers, states should insist — and the federal government should demand — that all new teachers have a major in the subject they expect to teach or preferably a strong educational background in two subjects, such as mathematics and music or history and literature. Every state should expect teachers to pass a rigorous examination in the subjects they will teach, as well as a general examination to demonstrate their literacy and numeracy.

Ravitch also calls for principals and superintendents to bring instructional experience to their work, not merely managerial talent.  “The principal is expected to evaluate teachers,” she notes.  ”If the principal is not a master teacher, he or she will not be able to perform the most crucial functions of the job.”  Similarly, superintendents who are not experienced educators “will not be qualified to select the best principals or the best curricula for their districts,” she writes.

We should stop using the term “failing schools” to describe schools with low test scores, counsels Ravitch, who also says schools should only be closed in rare circumstances.  ”We must break free of the NCLB mind-set that makes accountability synonymous with punishment. As we seek to rebuild our education system, we must improve the schools where performance is poor, not punish them. If we are serious about school reform, we will look for long-term solutions, not quick fixes,” she concludes.

What Business Needs From Schools: Character Education

by Robert Pondiscio
January 10th, 2010

“Teaching kids to be good is low hanging fruit with a lifetime payoff making for a productive society,” write a trio of high-ranking Wisconsin business executives in an opinion piece in the Milwaukee Journal-Sentinel calling for character education as a way to save public schools.

To people who run companies, honesty and punctuality are as important as computer literacy. Traits such as these are about respect for ourselves and others; they make up our character. Without character, quality work is almost impossible to produce no matter the number of employee incentives.

The trio, which includes the former CEO of Harley Davidson, represent a local chapter of the Character Education Partnership (CEP), a 17-year-old organization that encourages the teaching of ethical values  along with “supportive performance values” such as diligence, a strong work ethic, and perseverance.  Character education is essential, they write, and cost-effective. 

Curriculum experimentation is expensive and confusing to children. New equipment is expensive.  Instructing principals and teachers how to encourage children to exhibit good character, especially by modeling it, is not expensive.”

When teachers, students and school administrators respect each other, reading, math and science scores go up, the trio notes, without a change of curriculum, text books or the addition of expensive equipment.   “We’re not Luddites; we’re for technology, but if a school is in turmoil how will the students learn to use it?” they add.

Amen for this breath of fresh air from the business world, on a subject they know something about.  Personally, I was happy to read a prescription for schools from business executives that for once wasn’t about a lack of accountability, performance pay, how unions protect bad teachers, international competitiveness, the need innovation and to shatter the ”status quo.”

 

Whistleblowers Delight

by Robert Pondiscio
January 5th, 2010

Did anyone else get that remarkable email from the organizers of the Broader, Bolder Approach to Education yesterday?  The subject line read “BBA Needs Your Help.”  If you just hit delete, you missed a fascinating email.  BBA, which argues that test-driven accountability narrows the curriculum and creates test obsession in schools is asking teachers to submit examples of schools (presumably their own) that have suffered under strict accountability measures:

In a recent meeting, we advised Department of Education staff that their policy of identifying the lowest-performing 5% of schools in each state, in order to target these schools for massive intervention and “turnaround,” was bound to have adverse consequences if these schools were identified primarily by such test scores. We said that many schools that should be considered among the lowest performing schools would be missed if they artificially boosted their test scores at the expense of a balanced curriculum, by excessive test preparation activities and other gaming. And other schools that pursued a more balanced curriculum and attended to children’s long run achievement might falsely be identified as among the lowest-performing schools because they refused to engage in activities that artificially boosted test scores.

The letter, which doesn’t seem to appear on BBA’s website, notes DOE staff ”were not persuaded,” and asked the group to provide “examples of low-performing schools whose test scores have been artificially inflated by excessive test preparation and gaming, and better schools with very low scores but that were delivering a higher quality of instruction.”  The email, which carries the signatures of BBA organizers Helen Ladd, Pedro Noguera, and Tom Payzant, then asks recipients to identify such schools by name. 

Please include the name of the school, the name(s) of your source(s) of information, and other identifying information in your description. We will not initially provide all of this identifying information in the material we supply to the Department, but we have to be prepared to back up our claims by naming names if necessary.

It’s a bold move by BBA, although they might also consider sending along a copy of Linda Perlstein’s Tested.  I suspect they will find no shortage of schools that have muscled up on test prep and played games to boost test scores.  Whether teachers at those schools are willing to publicly say so is another matter. 

BBA is on shakier ground, I believe, in looking for good schools whose efforts don’t show up on test scores.  If a school is delivering a rigorous, well-rounded curriculum and “attending to children’s long run achievement” that should show up on test scores, assuming the effort is long-running, ongoing and well-implemented.

Softballs

by Robert Pondiscio
October 6th, 2009

A-Rus at This Week in Education looks at blogosphere reactions to Arne Duncan’s sit-down with Stephen Colbert and sees a pattern:  Everyone pans Colbert for lobbing softball questions at the Ed Secretary.  Huh?!?

Yes, more and more of us take our news and cues from The Daily Show, Colbert and late night comics.  I get it.  But have we really gotten to the point where we expect comedians to play Mike Wallace?  And are we really disappointed when they fail?

Why Send Kids To School?

by Robert Pondiscio
September 27th, 2009

“The single biggest problem in American education is that no one agrees on why we educate,” observes Diane Ravitch. ”Faced with this lack of consensus, policy makers define good education as higher test scores.”  Ravitch’s comments come in a forum published by the New York Times Magazine, which also features input from Tom Vander Ark, Geoffrey Canada, Charles Murray and others.  Ravitch writes:

Why do we educate? We educate because we want citizens who are capable of taking responsibility for their lives and for our democracy. We want citizens who understand how their government works, who are knowledgeable about the history of their nation and other nations. We need citizens who are thoroughly educated in science. We need people who can communicate in other languages. We must ensure that every young person has the chance to engage in the arts.

Reflecting on the theme of “How to Remake Education,” Vander Ark stumps for more attention to technology.  “By 2020, I believe most high-school students will do most of their learning online,” he writes.  “It shouldn’t take that long, but it will.”  Charles Murray argues we should “discredit the bachelor’s degree as a job credential”; while Canada believes we should lengthen the U.S. school year, which is “one of the shortest school years in the industrialized world.”

I’m with Diane. There is a clear failure of vision in American education at present, especially in poor, urban schools.  We have narrowed the definition of what it means to be educated in America.  When affluent parents choose a school for their children—when they enroll in a private school or buy a home near specific schools–reading scores are simply not part of their calculus.  It is assumed that in a good school every child will learn to read, and then read to learn.  That’s simply what schools do. When policy makers, education reformers and even teachers and administrators evaluate what makes schools in poor, urban neighborhoods good or bad, however, a single litmus test applies: performance on standardized reading tests.  For the children of the poor, a good grade on a state reading test has become what it means to be educated.  The contrast could not be clearer:  we set the finish line for other people’s children where we set the starting line for our own.

Observations on Observations

by Robert Pondiscio
August 27th, 2009

If you’re a teacher, would you rather be judged by a 200-page list of indicators of highly skilled teaching, or by a principal who shares your philosophy of teaching and learning, supports your approach and pretty much leaves you alone–but has the power to fire you at will? 

This question occurred to me after reading a long and excellent post by John Merrow over at Learning Matters on teacher observations. He concludes that the observation process is “changing for the better in some places, but that, unfortunately, it’s still mostly useless.”

In the old days, teachers closed their doors and did their thing, for better or for worse. As long as things were quiet, administrators [rarely] bothered to open the door to see what was going on, and teachers never watched each other at work. That’s changing, sometimes for better, sometimes for worse. In some schools today, teachers are actually expected to watch their peers teach, after which they share their analysis. In other schools, however, principals armed with lists sit in the back of the class checking off ‘behaviors’ and later give the teacher a ‘scorecard’ with her ‘batting average.’

“Whether these observations are diagnostic in nature and therefore designed to help teachers improve or a ‘gotcha’ game is the essential question,” Merrow perceptively observes.  Teacher observations, like test scores, will undoubtedly loom ever larger as the issue of teacher quality bubbles to the top of the nation’s education agenda.  Like test scores, there’s a lot to learn from observations.  And like test scores, we’re equally likely to learn the wrong lessons.

Of all the “best practices” that have migrated to education from the business world, the one that didn’t make the trip is the idea that good managers hire excellent people, empower them with real decision-making authority, then get out of their way.  The closest thing to that in education is “close your door and do your thing,” as Merrow puts it.  That goes against the grain in the Age of Accountability, but it is undeniable that for many excellent and experienced teachers and their students, it works perfectly.   And while that approach is endangered, it has not disappeared.  Nor should it.  The point of any accountability system should be to help bad schools and teachers look and act like good schools and teachers, not the opposite.  Our schools still have plenty of brilliant iconoclasts who do things their own way to great effect. 

For such  teachers nothing could be worse than “observation by checklist,” where the adminstration wants to see what it wants to see: aim and standard on the board?  Check.  Students sitting in groups?  Check.  Updated work on the bulletin board?  Check. A “print rich” environment in “kid-friendly language?” Check.   Ask why these items are important and you’ll invariably hear that it’s what the principal’s supervisor expects to see.  What they are indicative of is lost.  The consummate irony is this kind of evaluation seems rigorous, but it is more likely — much more likely — to create a civil service mentality than to foster excellence.  It’s another variation of the Cargo Cult Education phenomenon.   Teachers and administrators spend all their energy manufacturing the visible markers of learning, often not knowing (and after a while no longer caring) what the “indicators” indicate. 

Indeed, this is the thing the every teacher knows, that every armchair expert does not: it is simple (but time-consuming) to create an environment that gives all the appearances of being a high-functioning classroom and still be a lousy teacher.  Among the very first survival skills a new teacher learns, either through the advice of a kindly colleague or through a series of administrative reprimands, is the art of the dog and pony show.   In some schools, it’s the quid pro quo that earns you the right to close your door and practice your craft.  In more punitive environments, it’s the tail that wags the dog.   But the aim of observation-by-checklist is not great teaching, it’s plausible deniability–and it’s the enemy of accountability, for both teachers and administrators.  Miss Jones’ classroom demonstrates a high degree of student engagement and all of the indicators of high quality teaching, but her students are still not making progress.  Why? Miss Jones’ energy is misdirected.  She’s learning to play the game, not become a great teacher.  After a few years, she gets tired of it and quits.  Mediocrity wins again. 

The bottom line is that great teaching is like Potter Stewart’s definition of hard-core pornography.  It’s hard to define but you know it when you see it.  Unfortunately, that’s never going to cut it in our data-mad, accountability-obsessed age. 

So which would you rather?  Find a school and work with a principal who shares your philosophy and approach, trusts you and supports you, but has the power to fire you at will?  Or a school where your duties are codified to the letter, where you know what’s on the checklist and spend all of your time ‘working to rule‘ and playing “gotcha.”  Where are you going to be happiest and most productive?

Am I the only one who thinks this is what the teacher quality debate is really all about?