Second Thoughts on Pineapplegate

by Robert Pondiscio
May 4th, 2012

Writing in his TIME Magazine column, Andy “Eduwonk” Rotherham offers up a largely exculpatory take on Pineapplegate.  The media jumped all over a bowdlerized version of the test passage, he notes.  New York state officials should have been clearer in explaining that nothing makes its way onto standardized tests by accident.  And in the end, Andy writes, what is needed is “a more substantive conversation rather than a firestorm” over testing.

Very well, let’s have one.

In the unlikely event you haven’t heard, a minor media frenzy was ignited a few weeks back when the New York Daily News got hold of a surreal fable, loosely modeled on the familiar tale of the Tortoise and the Hare, which appeared on the just-administered New York State 8th grade reading test.  In the test passage, a talking pineapple challenges a hare to a foot race in front of a group of woodland creatures, loses the race (the pineapple’s lack of legs proving to be a fatal competitive disadvantage)  and gets eaten by the other animals.

Rotherham points out that the passage picked up by the paper was not the actual test passage, but a second-hand version plucked from an anti-testing website. “The passage the paper ran was so poorly written that it would indeed have been inexcusable,” he wrote.  Perhaps, but the correct passage wasn’t exactly a model of clarity and coherence either.  Indeed, the fable’s author mocked the decision by the testing company, Pearson, to create multiple choice questions about his story on a state test.  “As far as I am able to ascertain from my own work, there isn’t necessarily a specifically assigned meaning in anything,” Daniel Pinkwater told the Wall Street Journal. “That really is why it’s hilarious on the face of it that anybody creating a test would use a passage of mine, because I’m an advocate of nonsense. I believe that things mean things but they don’t have assigned meanings.”

Ultimately the real version of the test passage was released by the state to quiet the controversy.  But it did little to reverse the impression that this was a questionable measure of students’ ability.  Rotherham’s big “get” in Time is a memo from Pearson to New York State officials detailing the question’s review process as well as its use on other states’ tests as far back as 2004.  The message:  nothing to see here, folks.  Show’s over.  Go on back to your schools, sharpen those No. 2 pencils and get ready for more tests.

“Standardized tests are neither as bad as their critics make them out to be nor as good as they should be,” Rotherham concludes.  Perhaps, but they’re bad enough.  The principal problem, which Pineapplegate underscores vividly, is that we continue to insist on drawing conclusions about students’ reading ability based on a random, incoherent collection of largely meaningless passages concocted by test-makers utterly disconnected from what kids actually learn in school all day.  This actively incentivizes a form of educational malpractice, since reading tests reinforce the mistaken notion that reading comprehension is a transferable skill and that the subject matter is disconnected from comprehension.   But we know this is not the case as E.D. Hirsch and Dan Willingham have pointed out time and again, and as we have discussed on this blog repeatedly.

So this is not a simple case of an uproar based on bad information and sloppy damage control.  What Rotherham misses in a somewhat strident defense of standardized tests and testing is that we are suffering generally from a case of test fatigue. The entire edifice of reform rests on testing, and while the principle of accountability remains sound, the effects of testing on schools has proven to be deleterious, to be charitable. Thus the conditions were ripe for people to overreact to perceived absurdity in the tests. And that’s exactly what happened here.

Was the story was blown out of proportion by some people playing fast and loose with the facts?  Perhaps.  But the facts, once they became clear, were more than bad enough.

Erase to the Top

by Robert Pondiscio
March 28th, 2011

“On the 2009 reading test, for example, seventh-graders in one Noyes classroom averaged 12.7 wrong-to-right erasures per student on answer sheets; the average for seventh-graders in all D.C. schools on that test was less than 1. The odds are better for winning the Powerball grand prize than having that many erasures by chance, according to statisticians consulted by USA TODAY.”

A USA Today investigative piece looks at high erasure rates on standardized tests at Washington, DC’s Crosby S. Noyes Education Campus, which went from a school in need to one of DC’s ‘shining stars.’”  The report notes that three years ago, DC’s Office of the State Superintendent of Education asked test-maker McGraw-Hill to do “erasure analysis” after some schools showed big gains in in proficiency rates on April 2008 tests.  “Among 96 schools flagged for wrong-to-right erasures were eight of the 10 campuses where [DC Superintendent Michelle] Rhee handed out so-called TEAM awards ‘to recognize, reward and retain high-performing educators and support staff,’ as the district’s website says. Noyes was one of these.”

Moving the Chains

by Robert Pondiscio
September 30th, 2009

Football fans see it time and again:  It’s 4th down and short yardage.  An official standing 30 or 40 feet away from the play sees a running back hurl himself full throttle into a forest of 300-pound linemen and disappear beneath a collapsing pile of players, a football buried somewhere against his body.   Chaos everywhere, yet the official, with unquestioned authority places the ball he lost sight of on the exact spot on the ground where forward momentum stopped and calls for the chains.  Play stops and the fans grow quiet as a team of officials runs in from the sidelines and takes a precise-to-the-inch measurement of the ball’s location.  If the any part of the ball is beyond the plane of the outstretched chain, a first down is awarded.  The crowd goes wild. 

FIRST DOWN by MIKECNY.

Never mind that the linesman is merely estimating the ball’s position.  Never mind that the ten-yard length of chain was placed based on an eyeball approximation of where the series of downs began three plays ago.  Never mind that every play in the series of downs begins and ends with a best guess (the wide receiver was knocked out of bounds at about the 35-yard line) When it’s time to determine whether or not a first down is to be awarded, football is suddenly a game of inches

Games, playoff hopes, bowl bids and careers turn on a guess–or a series of guesses.  But no one seems to question it.  Call for the chains!  If you stop and think about it, this doesn’t make a lot of sense.  The answer however is simple: Don’t think about it.

Here are a few more things not to think about:

  • Writing in the New York Times, Todd Farley, the author of the book “Making the Grades: My Misadventures in the Standardized Testing Industry” describes getting a part-time, $8 an hour job scoring fourth-grade, state-wide reading comprehension tests after a five-minute interview.  “Arbitrary decision is the rule, not the exception,” he writes.  
  • “Bowen Elementary was part of what [Washington, DC] officials hailed as the success story of their 2008 standardized test results,” reports the Washington Post.  “But Bowen also had four classrooms where children erased wrong answers and replaced them with correct ones at abnormally high rates.”  The paper reports there were elevated numbers of erasures at six schools involving classrooms with 573 students.  CTB McGraw-Hill declared the data “inconclusive,” and no teachers or administrators have been accused of wrongdoing, the Post reports.  
  • In New York State seventh graders who answered just 44 percent of questions correctly on the state math test were given a passing grade. “Three years ago, the threshold for passing was 60 percent,” the New York Times reports. “In fact, students in every grade this year could slide by with fewer correct answers on the math test than in 2006.”
  • Teacher Diana Senechal recently described an experiment in which she was able to “pass” several standardized tests just by guessing and without even looking at the tests. 
  • “Policy makers define good education as higher test scores,” writes Diane Ravitch. “But students can get higher scores in reading and mathematics yet remain completely ignorant of science, the arts, civics, history, literature and foreign languages.” 

We know this.  We see it all around us, but like the football fan caught up in the arbitrary kabuki dance of the moving of the chains, we accept it, applaud it or moan about lousy spots, but the game goes on. 

“There must be a better way,” Pat Summerall, an N.F.L. veteran and broadcaster said in a recent New York Times article. “Because games are decided, careers are decided, on those measurements.”  He was talking about measuring for first downs.  “There’s a certain amount of drama that is involved with the chains,” said New York Giants president, John Mara in the same article. “Yes, it is subject to human error, just like anything else is. But I think it’s one of the traditions that we have in the game, and I don’t think any of us have felt a real compelling need to make a change.”

“With national standards will come national standardized tests, so it’s an especially good time to rethink how these exams are scored, and by whom,” Dana Goldstein sensibly observes at The American Prospect’s Tapped blog.  “Perhaps teachers and principals should be scoring tests, not $8 an hour part-timers. In that case it would be important, especially with the push for merit pay, to make sure teachers aren’t grading their own students’ tests, to decrease the temptation to engage in foul play.”

Like the theatrical measurement of a first down in football, we want to rely on precise measurements of an imprecise process to make high stakes decisions on everything from federal funding to merit pay to whether a teacher keeps his or her job at all.  “I understand that tests are far from perfect and that it is unfair to reduce the complex, nuanced work of teaching to a simple multiple choice exam,” Education Secretary Arne Duncan recently observed. 

Right.  It’s way more complicated than that. 

But it’s 4th down!  Call for the chains!  Take a measurement.  How else are we going to know?

The End of Education Reform

by Robert Pondiscio
September 21st, 2009

A remarkable speech by Chester Finn of the Fordham Institute is all the more remarkable for the lack of chatter it has generated in the edusphere.  Titled “Is It Time to Throw in the Towel on Education Reform?” the September 9 speech at Rice University notes a broad consensus on education reform that has existed for better than two decades is coming apart at the seams.  “The overriding goal of that consensus was to boost America’s academic achievement at the K-12 level,” Finn notes, and it gave rise to “a tsunami of standards-based reform.”

He cites several major developments contributing to the fraying of that consensus.  Among them: unhappiness with NCLB and a palpable backlash against testing that “goes to the heart of standards-based reform.”  On school choice, he points out, far too many charters and schools of choice have been “disappointingly mediocre.”  Then there are the results of the reform era:

Despite all the reforming, U.S. scores have remained essentially flat, graduation rates have remained essentially flat, and our international rankings have remained essentially flat. You can find some upward blips but you can also find downward blips. Big picture, over 25 years, is flat, flat, flat. In other words, all the reforming has yielded little or nothing by way of stronger outcomes.

Finn also cites “principled critiques by serious people” as another crack in the ed reform wall:

E.D. Hirsch’s new book may be its most cogent example, at least until Diane Ravitch’s next book emerges—of both standards-based reform and school choice on grounds that these structural changes neglect crucial issues of content and pedagogy—neglect what actually goes on in classrooms between teacher and learner—while narrowing the curriculum and weakening the common culture. 

 Has the reform consensus “outlived its usefulness?”  Finn compares American education to the situation the nation found itself in when the Articles of Confederation proved insufficient to the needs of the new nation.  “We may be at a similar stage with regard to our public-education system,” he notes. “Further tugging and kicking at it from the banks of the Potomac is not going to modernize it.”

I’m suggesting to you that American education today resembles America itself in 1785. The old arrangement isn’t working well enough and probably cannot be made to. A new constitution is needed. It’s in that sense that we should throw in the towel on education reform and think instead about reinvention.

 Checker briefly lists his ideas for “essential ingredients” of this new constitution including national standards and measures; portable statewide “weighted-student” financing; and the replacement of traditional school districts “with an array of virtual systems and regional or national operators (some of them technology-based).”

Ravitch Redux

by Robert Pondiscio
September 9th, 2009

Bridging Differences returns from summer hiatus today.  Deborah Meier and Diane Ravitch’s blog resumes with a Ravitch post that takes issue with the $5 billion “Race to the Top” fund.  “As I predicted on this blog, President Obama and Education Secretary Arne Duncan are now the spear carriers for the GOP’s education policies of choice and accountability,” Ravitch writes. 

What is extraordinary about these regulations is that they have no credible basis in research. They just happen to be the programs and approaches favored by the people in power. Under normal circumstances, the Department of Education would need congressional hearings and authorization to launch a program so sweeping and so sharply defined. Instead, they are using the “stimulus” money to impose their preferences, with no hearings and no congressional authorization.

Ravitch is also troubled by the administration’s push to tie teacher evaluations to student test scores.

I commend to our readers the response to the RTTT regulations by Professor Helen Ladd, an economist who has studied teacher evaluation for many years, as well as the one by Paul Barton, who has studied education issues for many years. What both of these responses clearly demonstrate is that there is no research basis for the priorities favored by Secretary Duncan.

“This will be an interesting year,” Ravitch concludes. ”But also a very dangerous year for American public education.”

More Diane Ravitch:  She weighs in on the NYC school report card controversy in today’s New York Post

Social Promotion? Easy as A, B, C!

by Robert Pondiscio
August 18th, 2009

Can you earn a promotion to the next grade in New York by simply guessing the answers on state tests?   It’s easy as A, B, C according to a provocative experiment by former Core Knowledge teacher Diana Senechal.   

In a call for tougher tests in the New York Post last week, Diane Ravitch revealed that the points needed to earn a “Level 2″ — the lowest “passing” score on the state’s tests–have dropped dramatically.  On the 6th grade English language-arts test, for example, the cutoff to earn a Level 2 in sixth grade dropped from 41 percent of the points in 2006 to just 17.9 percent in 2009.  “Ending social promotion, as the [New York City] rightly wants to do, is thus meaningless, because students can reach Level 2 by just guessing,” Ravitch concluded.

Struck by Ravitch’s observation, Senechal tried an experiment to see if it’s possible to pass the test by simply guessing.  She posts the results over at Gotham Schools

I first tried my experiment with the sixth grade ELA test. I “guessed” all the answers on the multiple-choice portion and left the written portions blank. Or, rather, I didn’t “guess,” but filled in the answers as follows: A, B, C, D, A, B, C, D, and so on, all the way through the 26 questions. I didn’t read one of them.

Naturally, Senechal got a zero on the written portion of the test.  But her multiple choice guesswork earned 12 out of 39 “raw points” and a scale score of 622–a rock-solid “2″ on the state’s four-point system.  A “2″ is described as “approaching grade level” and good enough to earn promotion to the next grade.  “I got a 2 without looking at a single test question or writing a single word,” she writes.   Repeating the experiment with the 7th grade math test, Senechal also scored a 2 “without solving a single math problem, or even looking at one.”

While this approach does not result in a 2 for all the tests, it comes a bit too close for comfort, and another guessing system might work. A fifth grader told me that his father had told him, “Just mark ‘C’ for all of the answers, and you will pass.” On the fifth grade ELA test, this would indeed have resulted in a 2.  Yes, it is possible to guess your way to promotion. You may not even have to look at the questions or write a word on the written sections.

“It may not be called social promotion, but it amounts to the same thing,” concludes Senechal, a frequent contributor to the Core Knowledge Blog.  “You do not need to know or understand much to move along.”

African-American Students Report to the Gym

by Robert Pondiscio
April 30th, 2009

So now it’s come to this.

Students at a Sacramento-area high school attended standardized test pep rallies — er, sorry…Heritage Assemblies – organized by race to pump up each ethnic group to take state tests.  “Students could go to any rally they wanted,” the Sacramento Bee reports, ”but the gatherings were designated for specific races – African Americans in the gym, Pacific Islanders in the theater, Latinos in the multipurpose room.”

The paper describes a scene in the gym at Laguna Creek High School, where students gathered before a large outline of Africa on the wall. “Last year we scored the highest percentage increase of any group,” Vice Principal Hasan Abdulmalik hollered at the crowd.

Lovely.

Laguna Creek High School Principal Doug Craig said dividing the students by race allowed staff to talk about test scores without making any one ethnic group feel singled out in a negative manner. “Is it racist? I don’t believe it is,” Craig tells the paper, which reports the practice of holding race-specific test prep rallies has become more common in California.  

Gathering and reporting data based on ethnic groups is one of the few unambiguous wins of the NCLB era.  It’s pushed the achievement gap to the front of our education agenda.  But I’m not sure holding “heritage rallies” even rises to the level of well-intentioned but wrong-headed.  At best, it’s yet another example of how schools are putting their problems–and their desperation– on the backs of kids. And a particularly disturbing example at that.

Update:  I was remiss in not tipping my hat to Anthony Rebora, who brought this item to my attention via his forum at Teacher Magazine.

E.D. Hirsch on “Reading Test Dummies”

by Robert Pondiscio
April 16th, 2009

Earlier this week, Kathleen Dunn of Wisconsin Public Radio hosted E.D. Hirsch for an hour-long discussion of his recent New York Times op-ed, “Reading Test Dummies,” which called for aligning the reading passages on standardized tests with specific curricular content taught in each grade.  The interview is now archived on WPR’s website here.

Attendance Is Not On The Test

by Robert Pondiscio
October 21st, 2008

More than 90,000 of New York City’s elementary school students–20 percent–missed at least a month of classes during the last school year, according to a new report from the Center for New York City Affairs at The New School.

In the early grades, attendance is a strong predictor of long-term success. National research suggests that chronic absenteeism in the early grades sets the stage for school failure later on. Children who miss a large number of school days in kindergarten or first grade tend to have lower levels of academic achievement throughout their school careers. Sadly, there are high levels of chronic absenteeism in New York City elementary schools, particularly in low-income neighborhoods.

It’s great to see this issue getting some attention, but forgive me if I’m utterly unsurprised, and a little disgusted.  The New York Times calls chronic absenteeism an “invisible problem” but it’s anything but to teachers in New York’s most blighted inner city neighborhoods.  Frankly, it’s also another unintended consequence of system in which The Test is the alpha and omega.  In my South Bronx elementary school we regularly promoted students who missed dozens of school days, as long as they passed — or even came close to passing – a single standardized test.  In a particularly acute case, I fought unsuccessfully to have one of my 5th graders held over who missed nearly 100 school days.  He received a 1 (below grade level) on his state math test and a 2 (“approaching” grade level) on his ELA exam and was passed without even having to attend summer school.  As long as he scored a 2 or better on either of the tests, I was told, he had to be promoted.  God help that kid.  Three years later, I still get angry thinking about it.  

In theory, I asked an administrator, could a child come to school only on the day of the state test, pass, and still be promoted?  It was a rhetorical question.  The answer was sitting in my classroom.  Occasionally.

Principal Apologizes for “Excellent” Rating

by Robert Pondiscio
July 23rd, 2008

The principal of Rocky River Middle School in Ohio is sorry.

His school made AYP, earned an “excellent” rating from the state, and passed the 2008 Ohio Achievement. But principal David Root gave Cleveland Plain Dealer columnist Regina Brett a remarkable two-page, single-spaced apology, addressed to students, staff and citizens of Rocky River, detailing the cost of those accomplishments. Among the things Root is sorry for:

  • That he spent thousands of tax dollars on test materials, practice tests, postage and costs for test administration.
  • That his teachers spent less time teaching American history because most of the social studies test questions are about foreign countries.
  • That he didn’t suspend a student for assaulting another because that student would have missed valuable test days.
  • For pulling children away from art, music and gym, classes they love, so they could take test-taking strategies.
  • That he has to give a test where he can’t clarify any questions, make any comments to help in understanding or share the results so students can actually learn from their mistakes.
  • That the integrity of his teachers is publicly tied to one test.
  • For making decisions on assemblies, field trips and musical performances based on how that time away from reading, math, social studies and writing will impact state test results.
  • For arranging for some students to be labeled “at risk” in front of their peers and put in small groups so the school would have a better chance of passing tests.
  • For making his focus as a principal no longer helping his staff teach students but helping them teach test indicators.

“We don’t teach kids anymore,” Root, a 24-year veteran principal, tells the paper. “We teach test-taking skills. We all teach to the test. I long for the days when we used to teach kids.”