Second Thoughts on Pineapplegate

by Robert Pondiscio
May 4th, 2012

Writing in his TIME Magazine column, Andy “Eduwonk” Rotherham offers up a largely exculpatory take on Pineapplegate.  The media jumped all over a bowdlerized version of the test passage, he notes.  New York state officials should have been clearer in explaining that nothing makes its way onto standardized tests by accident.  And in the end, Andy writes, what is needed is “a more substantive conversation rather than a firestorm” over testing.

Very well, let’s have one.

In the unlikely event you haven’t heard, a minor media frenzy was ignited a few weeks back when the New York Daily News got hold of a surreal fable, loosely modeled on the familiar tale of the Tortoise and the Hare, which appeared on the just-administered New York State 8th grade reading test.  In the test passage, a talking pineapple challenges a hare to a foot race in front of a group of woodland creatures, loses the race (the pineapple’s lack of legs proving to be a fatal competitive disadvantage)  and gets eaten by the other animals.

Rotherham points out that the passage picked up by the paper was not the actual test passage, but a second-hand version plucked from an anti-testing website. “The passage the paper ran was so poorly written that it would indeed have been inexcusable,” he wrote.  Perhaps, but the correct passage wasn’t exactly a model of clarity and coherence either.  Indeed, the fable’s author mocked the decision by the testing company, Pearson, to create multiple choice questions about his story on a state test.  “As far as I am able to ascertain from my own work, there isn’t necessarily a specifically assigned meaning in anything,” Daniel Pinkwater told the Wall Street Journal. “That really is why it’s hilarious on the face of it that anybody creating a test would use a passage of mine, because I’m an advocate of nonsense. I believe that things mean things but they don’t have assigned meanings.”

Ultimately the real version of the test passage was released by the state to quiet the controversy.  But it did little to reverse the impression that this was a questionable measure of students’ ability.  Rotherham’s big “get” in Time is a memo from Pearson to New York State officials detailing the question’s review process as well as its use on other states’ tests as far back as 2004.  The message:  nothing to see here, folks.  Show’s over.  Go on back to your schools, sharpen those No. 2 pencils and get ready for more tests.

“Standardized tests are neither as bad as their critics make them out to be nor as good as they should be,” Rotherham concludes.  Perhaps, but they’re bad enough.  The principal problem, which Pineapplegate underscores vividly, is that we continue to insist on drawing conclusions about students’ reading ability based on a random, incoherent collection of largely meaningless passages concocted by test-makers utterly disconnected from what kids actually learn in school all day.  This actively incentivizes a form of educational malpractice, since reading tests reinforce the mistaken notion that reading comprehension is a transferable skill and that the subject matter is disconnected from comprehension.   But we know this is not the case as E.D. Hirsch and Dan Willingham have pointed out time and again, and as we have discussed on this blog repeatedly.

So this is not a simple case of an uproar based on bad information and sloppy damage control.  What Rotherham misses in a somewhat strident defense of standardized tests and testing is that we are suffering generally from a case of test fatigue. The entire edifice of reform rests on testing, and while the principle of accountability remains sound, the effects of testing on schools has proven to be deleterious, to be charitable. Thus the conditions were ripe for people to overreact to perceived absurdity in the tests. And that’s exactly what happened here.

Was the story was blown out of proportion by some people playing fast and loose with the facts?  Perhaps.  But the facts, once they became clear, were more than bad enough.

Moving the Goalposts Back

by Robert Pondiscio
July 20th, 2010

School superintendents in some New York cities are complaining that changes to state tests that would lower passing rates and toughen the definition of proficiency are “akin to moving goalposts.”  “We’ve lost sight of the purpose of the test — it’s supposed to show you’ve mastered a certain skill at a certain time,” Daniel G. Lowengard, the superintendent in Syracuse tells the New York Times.  “I think it’s unfair to teachers to say thank you very much, you’ve been doing this work for the last three or four years, and now that your kids are passing, all of sudden we’re going to call a B a C and call a C a D.”

No.  Exactly wrong. The point is that kids have NOT mastered a certain skill at a certain time.  We’ve just been pretending they did.   The goalposts aren’t moving, they’ve been creeping in for years.  What David Steiner and Co. are talking about is moving them back into the endzone where they belong and (hopefully) bolting them to the ground.  If we must have football analogies, this is the proper one:  We’ve been telling kids for years they’re nailing kicks from 60 yards out and are NFL material.  Only it turns out the goalposts were merely 20 yards away.   Telling someone they’ve got the tools when they don’t is not just misleading but cruel.

David Steiner Gets It

by Robert Pondiscio
July 14th, 2010

Keep an eye on New York State education commissioner David Steiner, who is gearing up to implement a long overdue reform: establishing a link between test scores and college readiness. 

Harvard’s Daniel Koretz, at Steiner’s urging, has been looking at the correlation between New York’s eighth-grade test scores and high school Regents exam scores. Notes the Buffalo News:  ”The conclusion: Students in New York State are moving through elementary, middle and high school with test scores they believe to be adequate, but once they get to college, they find they are not prepared.”  That’s not a complete shock given the boxcar numbers of college freshman who need remediation once they arrive on campus.  But the New York Post’s Yoav Gonen points out what will surely be the most repeated fact from Koretz’s forthcoming study: eighth-graders who score a 3 out of 4 on state math and reading tests have just a 52 percent chance of graduating high school, even though they’ve been told they’re on track.

Let that rattle around inside your head for a moment:  A child who is deemed proficient in 8th grade has a chance only slightly better than a coin toss of graduating high school just four years later.   “We’ve been calling that ‘proficient,’ ” state Board of Regents Chancellor Merryl Tisch told The Post’s editorial board. “We were giving out misleading information.”

Gee, ya think?

The study is to be released Monday, but anyone who has taught in New York in the last several years can’t be surprised.  For years, I saw 5th graders come into my Bronx classroom who were ostensibly on grade level yet demonstrated little command of basic arithmetic. That was plenty persuasive that all that glitters isn’t gold.

Steiner’s insistence that test scores should actually mean something is clearly going to rattle some cages, and prompt a long hard look at where school districts in New York have made real gains and where they haven’t.  Buffalo’s school superintendent blasted Steiner and his deputy John King last week for focusing on more rigorous tests.  ”I think they’re two people who don’t know what they’re doing,” James A. Williams told the Buffalo News. “A more rigorous test is not going to improve student achievement. It’s not going to improve the graduation rate. I think it’s ridiculous.”

I don’t follow Williams’ complaint.   By my read, Steiner isn’t talking about testing our way to proficiency.  He’s talking about how test scores should be indicative of real-world proficiency.  As I’ve argued in this space before, if we’re going to insist on viewing everything in education through the prism of test scores, those scores have to be meaningful and indicative of real-world proficiency.  Steiner, King and Tisch deserve all the credit in the world for taking this on.

Annals of Lying to Children

by Robert Pondiscio
June 7th, 2010

“I think we are lying to children and families when we tell children that they are meeting standards and, in fact, they are woefully unprepared to be successful in high school and have almost no chance of going to a good university and being successful”  — Secretary of Education, Arne Duncan

Students taking New York State math tests earned partial credit for wrong answers while some “got credit for no answer at all,” reports the New York Post, which has several examples of generous partial credit given for wrong answers from a scoring guide it obtained for the state’s most recent 4th grade math test:

  • A kid who answers that a 2-foot-long skateboard is 48 inches long gets half-credit for adding 24 and 24 instead of the correct 12 plus 12.
  • A miscalculation that 28 divided by 14 equals 4 instead of 2 is “partially correct” if the student uses the right method to verify the wrong answer.
  • Setting up a division problem to find one-fifth of $400, but not solving the problem — and leaving the answer blank — gets half-credit.
  • A kid who subtracts 57 cents from three quarters for the right change and comes up with 15 cents instead of 18 cents still gets half-credit.
  • A student who figures the numbers of books in 35 boxes of 10 gets half-credit despite messed-up multiplication that yields the wrong answer, 150 instead of 350.

The “holistic rubrics,” according to the paper, give points “if a kid’s attempt at an answer reflects a ‘partial understanding’ of the math concept, ‘addresses some element of the task correctly,’ or uses the ‘appropriate process.’ to arrive at a wrong solution.”

Teachers will almost certainly agree that there is much merit in partial credit for partial understanding.  The question is how much.  The Post quotes a Brooklyn teacher who says of the grading rubric, “You feel like you’re being forced to cheat.”  A New York State Ed Department spokesperson defends the rubric, but it’s hard to imagine recently appointed chairman David Steiner being pleased with it.  As Sol Stern reported recently, Steiner and the chancellor of the Board of Regents, Merryl Tisch, have brought in assessment expert Daniel Koretz to conduct an independent audit of state reading and math tests.  Critics have levelled numerous charges of dumbed-down tests and lowered cut scores.

The most serious disservice resulting from such clear and obvious grade inflation is the misimpression it creates among the most vulnerable students and parents.  Families who are not critical consumers of education–who may not be well-educated themselves but have ambitions for their children–reasonably assume that “on grade level” means, well, on grade level and that their child is on track to graduate from high school and go to college.  Creating such a misimpression is unforgivable.