Tag Archive for 'standardized testing'

Moving the Chains

Published by Robert Pondiscio on September 30, 2009 in Assessment and Testing. 3 Comments

Football fans see it time and again:  It’s 4th down and short yardage.  An official standing 30 or 40 feet away from the play sees a running back hurl himself full throttle into a forest of 300-pound linemen and disappear beneath a collapsing pile of players, a football buried somewhere against his body.   Chaos everywhere, yet the official, with unquestioned authority places the ball he lost sight of on the exact spot on the ground where forward momentum stopped and calls for the chains.  Play stops and the fans grow quiet as a team of officials runs in from the sidelines and takes a precise-to-the-inch measurement of the ball’s location.  If the any part of the ball is beyond the plane of the outstretched chain, a first down is awarded.  The crowd goes wild. 

FIRST DOWN by MIKECNY.

Never mind that the linesman is merely estimating the ball’s position.  Never mind that the ten-yard length of chain was placed based on an eyeball approximation of where the series of downs began three plays ago.  Never mind that every play in the series of downs begins and ends with a best guess (the wide receiver was knocked out of bounds at about the 35-yard line) When it’s time to determine whether or not a first down is to be awarded, football is suddenly a game of inches. 

Games, playoff hopes, bowl bids and careers turn on a guess–or a series of guesses.  But no one seems to question it.  Call for the chains!  If you stop and think about it, this doesn’t make a lot of sense.  The answer however is simple: Don’t think about it.

Here are a few more things not to think about:

  • Writing in the New York Times, Todd Farley, the author of the book “Making the Grades: My Misadventures in the Standardized Testing Industry” describes getting a part-time, $8 an hour job scoring fourth-grade, state-wide reading comprehension tests after a five-minute interview.  “Arbitrary decision is the rule, not the exception,” he writes.  
  • “Bowen Elementary was part of what [Washington, DC] officials hailed as the success story of their 2008 standardized test results,” reports the Washington Post.  “But Bowen also had four classrooms where children erased wrong answers and replaced them with correct ones at abnormally high rates.”  The paper reports there were elevated numbers of erasures at six schools involving classrooms with 573 students.  CTB McGraw-Hill declared the data “inconclusive,” and no teachers or administrators have been accused of wrongdoing, the Post reports.  
  • In New York State seventh graders who answered just 44 percent of questions correctly on the state math test were given a passing grade. “Three years ago, the threshold for passing was 60 percent,” the New York Times reports. “In fact, students in every grade this year could slide by with fewer correct answers on the math test than in 2006.”
  • Teacher Diana Senechal recently described an experiment in which she was able to “pass” several standardized tests just by guessing and without even looking at the tests. 
  • “Policy makers define good education as higher test scores,” writes Diane Ravitch. “But students can get higher scores in reading and mathematics yet remain completely ignorant of science, the arts, civics, history, literature and foreign languages.” 

We know this.  We see it all around us, but like the football fan caught up in the arbitrary kabuki dance of the moving of the chains, we accept it, applaud it or moan about lousy spots, but the game goes on. 

“There must be a better way,” Pat Summerall, an N.F.L. veteran and broadcaster said in a recent New York Times article. “Because games are decided, careers are decided, on those measurements.”  He was talking about measuring for first downs.  “There’s a certain amount of drama that is involved with the chains,” said New York Giants president, John Mara in the same article. “Yes, it is subject to human error, just like anything else is. But I think it’s one of the traditions that we have in the game, and I don’t think any of us have felt a real compelling need to make a change.”

“With national standards will come national standardized tests, so it’s an especially good time to rethink how these exams are scored, and by whom,” Dana Goldstein sensibly observes at The American Prospect’s Tapped blog.  “Perhaps teachers and principals should be scoring tests, not $8 an hour part-timers. In that case it would be important, especially with the push for merit pay, to make sure teachers aren’t grading their own students’ tests, to decrease the temptation to engage in foul play.”

Like the theatrical measurement of a first down in football, we want to rely on precise measurements of an imprecise process to make high stakes decisions on everything from federal funding to merit pay to whether a teacher keeps his or her job at all.  “I understand that tests are far from perfect and that it is unfair to reduce the complex, nuanced work of teaching to a simple multiple choice exam,” Education Secretary Arne Duncan recently observed. 

Right.  It’s way more complicated than that. 

But it’s 4th down!  Call for the chains!  Take a measurement.  How else are we going to know?

The End of Education Reform

Published by Robert Pondiscio on September 21, 2009 in No category. 11 Comments

A remarkable speech by Chester Finn of the Fordham Institute is all the more remarkable for the lack of chatter it has generated in the edusphere.  Titled “Is It Time to Throw in the Towel on Education Reform?” the September 9 speech at Rice University notes a broad consensus on education reform that has existed for better than two decades is coming apart at the seams.  “The overriding goal of that consensus was to boost America’s academic achievement at the K-12 level,” Finn notes, and it gave rise to “a tsunami of standards-based reform.”

He cites several major developments contributing to the fraying of that consensus.  Among them: unhappiness with NCLB and a palpable backlash against testing that “goes to the heart of standards-based reform.”  On school choice, he points out, far too many charters and schools of choice have been “disappointingly mediocre.”  Then there are the results of the reform era:

Despite all the reforming, U.S. scores have remained essentially flat, graduation rates have remained essentially flat, and our international rankings have remained essentially flat. You can find some upward blips but you can also find downward blips. Big picture, over 25 years, is flat, flat, flat. In other words, all the reforming has yielded little or nothing by way of stronger outcomes.

Finn also cites “principled critiques by serious people” as another crack in the ed reform wall:

E.D. Hirsch’s new book may be its most cogent example, at least until Diane Ravitch’s next book emerges—of both standards-based reform and school choice on grounds that these structural changes neglect crucial issues of content and pedagogy—neglect what actually goes on in classrooms between teacher and learner—while narrowing the curriculum and weakening the common culture. 

 Has the reform consensus “outlived its usefulness?”  Finn compares American education to the situation the nation found itself in when the Articles of Confederation proved insufficient to the needs of the new nation.  “We may be at a similar stage with regard to our public-education system,” he notes. “Further tugging and kicking at it from the banks of the Potomac is not going to modernize it.”

I’m suggesting to you that American education today resembles America itself in 1785. The old arrangement isn’t working well enough and probably cannot be made to. A new constitution is needed. It’s in that sense that we should throw in the towel on education reform and think instead about reinvention.

 Checker briefly lists his ideas for “essential ingredients” of this new constitution including national standards and measures; portable statewide “weighted-student” financing; and the replacement of traditional school districts “with an array of virtual systems and regional or national operators (some of them technology-based).”

Ravitch Redux

Published by Robert Pondiscio on September 9, 2009 in Education News. 14 Comments

Bridging Differences returns from summer hiatus today.  Deborah Meier and Diane Ravitch’s blog resumes with a Ravitch post that takes issue with the $5 billion “Race to the Top” fund.  “As I predicted on this blog, President Obama and Education Secretary Arne Duncan are now the spear carriers for the GOP’s education policies of choice and accountability,” Ravitch writes. 

What is extraordinary about these regulations is that they have no credible basis in research. They just happen to be the programs and approaches favored by the people in power. Under normal circumstances, the Department of Education would need congressional hearings and authorization to launch a program so sweeping and so sharply defined. Instead, they are using the “stimulus” money to impose their preferences, with no hearings and no congressional authorization.

Ravitch is also troubled by the administration’s push to tie teacher evaluations to student test scores.

I commend to our readers the response to the RTTT regulations by Professor Helen Ladd, an economist who has studied teacher evaluation for many years, as well as the one by Paul Barton, who has studied education issues for many years. What both of these responses clearly demonstrate is that there is no research basis for the priorities favored by Secretary Duncan.

“This will be an interesting year,” Ravitch concludes. ”But also a very dangerous year for American public education.”

More Diane Ravitch:  She weighs in on the NYC school report card controversy in today’s New York Post

Social Promotion? Easy as A, B, C!

Published by Robert Pondiscio on August 18, 2009 in Assessment and Testing, Core Knowledge and Educational Policy. 1 Comment

Can you earn a promotion to the next grade in New York by simply guessing the answers on state tests?   It’s easy as A, B, C according to a provocative experiment by former Core Knowledge teacher Diana Senechal.   

In a call for tougher tests in the New York Post last week, Diane Ravitch revealed that the points needed to earn a “Level 2″ — the lowest “passing” score on the state’s tests–have dropped dramatically.  On the 6th grade English language-arts test, for example, the cutoff to earn a Level 2 in sixth grade dropped from 41 percent of the points in 2006 to just 17.9 percent in 2009.  “Ending social promotion, as the [New York City] rightly wants to do, is thus meaningless, because students can reach Level 2 by just guessing,” Ravitch concluded.

Struck by Ravitch’s observation, Senechal tried an experiment to see if it’s possible to pass the test by simply guessing.  She posts the results over at Gotham Schools. 

I first tried my experiment with the sixth grade ELA test. I “guessed” all the answers on the multiple-choice portion and left the written portions blank. Or, rather, I didn’t “guess,” but filled in the answers as follows: A, B, C, D, A, B, C, D, and so on, all the way through the 26 questions. I didn’t read one of them.

Naturally, Senechal got a zero on the written portion of the test.  But her multiple choice guesswork earned 12 out of 39 “raw points” and a scale score of 622–a rock-solid “2″ on the state’s four-point system.  A “2″ is described as “approaching grade level” and good enough to earn promotion to the next grade.  “I got a 2 without looking at a single test question or writing a single word,” she writes.   Repeating the experiment with the 7th grade math test, Senechal also scored a 2 “without solving a single math problem, or even looking at one.”

While this approach does not result in a 2 for all the tests, it comes a bit too close for comfort, and another guessing system might work. A fifth grader told me that his father had told him, “Just mark ‘C’ for all of the answers, and you will pass.” On the fifth grade ELA test, this would indeed have resulted in a 2.  Yes, it is possible to guess your way to promotion. You may not even have to look at the questions or write a word on the written sections.

“It may not be called social promotion, but it amounts to the same thing,” concludes Senechal, a frequent contributor to the Core Knowledge Blog.  “You do not need to know or understand much to move along.”

African-American Students Report to the Gym

Published by Robert Pondiscio on April 30, 2009 in accountability. 8 Comments

So now it’s come to this.

Students at a Sacramento-area high school attended standardized test pep rallies — er, sorry…Heritage Assemblies – organized by race to pump up each ethnic group to take state tests.  “Students could go to any rally they wanted,” the Sacramento Bee reports, ”but the gatherings were designated for specific races – African Americans in the gym, Pacific Islanders in the theater, Latinos in the multipurpose room.”

The paper describes a scene in the gym at Laguna Creek High School, where students gathered before a large outline of Africa on the wall. “Last year we scored the highest percentage increase of any group,” Vice Principal Hasan Abdulmalik hollered at the crowd.

Lovely.

Laguna Creek High School Principal Doug Craig said dividing the students by race allowed staff to talk about test scores without making any one ethnic group feel singled out in a negative manner. “Is it racist? I don’t believe it is,” Craig tells the paper, which reports the practice of holding race-specific test prep rallies has become more common in California.  

Gathering and reporting data based on ethnic groups is one of the few unambiguous wins of the NCLB era.  It’s pushed the achievement gap to the front of our education agenda.  But I’m not sure holding “heritage rallies” even rises to the level of well-intentioned but wrong-headed.  At best, it’s yet another example of how schools are putting their problems–and their desperation– on the backs of kids. And a particularly disturbing example at that.

Update:  I was remiss in not tipping my hat to Anthony Rebora, who brought this item to my attention via his forum at Teacher Magazine.

E.D. Hirsch on “Reading Test Dummies”

Published by Robert Pondiscio on April 16, 2009 in Assessment and Testing. 1 Comment

Earlier this week, Kathleen Dunn of Wisconsin Public Radio hosted E.D. Hirsch for an hour-long discussion of his recent New York Times op-ed, “Reading Test Dummies,” which called for aligning the reading passages on standardized tests with specific curricular content taught in each grade.  The interview is now archived on WPR’s website here.

Attendance Is Not On The Test

Published by Robert Pondiscio on October 21, 2008 in Education Practice, Educational Policy and accountability. 8 Comments

More than 90,000 of New York City’s elementary school students–20 percent–missed at least a month of classes during the last school year, according to a new report from the Center for New York City Affairs at The New School.

In the early grades, attendance is a strong predictor of long-term success. National research suggests that chronic absenteeism in the early grades sets the stage for school failure later on. Children who miss a large number of school days in kindergarten or first grade tend to have lower levels of academic achievement throughout their school careers. Sadly, there are high levels of chronic absenteeism in New York City elementary schools, particularly in low-income neighborhoods.

It’s great to see this issue getting some attention, but forgive me if I’m utterly unsurprised, and a little disgusted.  The New York Times calls chronic absenteeism an “invisible problem” but it’s anything but to teachers in New York’s most blighted inner city neighborhoods.  Frankly, it’s also another unintended consequence of system in which The Test is the alpha and omega.  In my South Bronx elementary school we regularly promoted students who missed dozens of school days, as long as they passed — or even came close to passing – a single standardized test.  In a particularly acute case, I fought unsuccessfully to have one of my 5th graders held over who missed nearly 100 school days.  He received a 1 (below grade level) on his state math test and a 2 (”approaching” grade level) on his ELA exam and was passed without even having to attend summer school.  As long as he scored a 2 or better on either of the tests, I was told, he had to be promoted.  God help that kid.  Three years later, I still get angry thinking about it.  

In theory, I asked an administrator, could a child come to school only on the day of the state test, pass, and still be promoted?  It was a rhetorical question.  The answer was sitting in my classroom.  Occasionally.

Principal Apologizes for “Excellent” Rating

Published by Robert Pondiscio on July 23, 2008 in Assessment and Testing and Curriculum. 15 Comments

The principal of Rocky River Middle School in Ohio is sorry.

His school made AYP, earned an “excellent” rating from the state, and passed the 2008 Ohio Achievement. But principal David Root gave Cleveland Plain Dealer columnist Regina Brett a remarkable two-page, single-spaced apology, addressed to students, staff and citizens of Rocky River, detailing the cost of those accomplishments. Among the things Root is sorry for:

  • That he spent thousands of tax dollars on test materials, practice tests, postage and costs for test administration.
  • That his teachers spent less time teaching American history because most of the social studies test questions are about foreign countries.
  • That he didn’t suspend a student for assaulting another because that student would have missed valuable test days.
  • For pulling children away from art, music and gym, classes they love, so they could take test-taking strategies.
  • That he has to give a test where he can’t clarify any questions, make any comments to help in understanding or share the results so students can actually learn from their mistakes.
  • That the integrity of his teachers is publicly tied to one test.
  • For making decisions on assemblies, field trips and musical performances based on how that time away from reading, math, social studies and writing will impact state test results.
  • For arranging for some students to be labeled “at risk” in front of their peers and put in small groups so the school would have a better chance of passing tests.
  • For making his focus as a principal no longer helping his staff teach students but helping them teach test indicators.

“We don’t teach kids anymore,” Root, a 24-year veteran principal, tells the paper. “We teach test-taking skills. We all teach to the test. I long for the days when we used to teach kids.”

Teaching to the Tex

Published by Robert Pondiscio on July 21, 2008 in Assessment and Testing. 1 Comment

A section of the Texas Assessment of Knowledge and Skills (TAKS) is coming under scrutiny.  Even Texas’ best students struggle with a section of the test that asks students to express themselves and back up their claims with evidence, revealing either faulty tests or preparation.

Three short-response questions require students to stretch their brains by generating clear, reasonable ideas from a reading selection, the Dallas Morning News reports.  Then they must support those ideas with evidence from the text in a well-written response.  ”Students are passing the ninth-, 10th- and 11th-grade language arts TAKS at higher rates than ever, the paper notes. “Some even post near-perfect passing rates. But on the short-response portion, fewer than half of North Texas students pass.”

Texas Education Agency officials say the short-response questions provide a better window into how well students can think, communicate and write.  ”This paints a much different picture for teachers and parents than the multiple-choice test,” Victoria Young, a testing official with TEA tells the paper. “You’re finding out two very different things about kids.”  Richard Kouri of the Texas State Teachers Association said curriculum doesn’t have the depth it used to because teachers are pulled in so many different directions by the TAKS demands.

Here’s the scoring rubric for the short-answer reading section of the test.  Seems a reasonable set of tasks for high school students.