Archive for the 'Assessment and Testing' Category

Give Me Harvard or Give Me Death

Published by Robert Pondiscio on November 10, 2009 in Assessment and Testing, Parents and accountability. 0 Comments

Parental anxiety is ruining playtime, notes the Washington Post’s Valerie Strauss.  It’s not news that lots of preschool parents have become “super-anxious trying to give their kids a leg up on kindergarten,” Strauss writes at The Answer Sheet.  “But I didn’t realize just how nutty things had become until I talked to several dozen preschool program directors.”

Among the examples she cites: parents begging school directors to let their 1 1/2 -year-olds into programs for 2-year-olds “because Danny and Olivia are so incredibly advanced”; demanding to know why 2-year-olds aren’t being given the alphabet to copy over and over and memorize; and enrolling their kids in so many activities that three year olds fall asleep at their desks.

“People! This is the only time your child has to be a child!” she writes.  I was in complete agreement until I got to this line:  “The reason for all of this is No Child Left Behind, which has pushed curriculum down into the earliest grades and put the focus on high-stakes standardized tests that start as early as third grade.”

“I’m sorry, but blaming NCLB for elite parents pushing preschoolers too hard on academics and activities is BS,” says New America’s Sara Mead on Twitter.  Agreed.  A generation ago, New York Magazine wrote a cover story about the fierce competition among Manhattan parents to get Danny and Olivia into just the right preschool, just the right prep school, just the right college–and the relentless pressure on even the youngest kids.  The legendary cover line: “Give Me Harvard or Give Me Death.”

There’s plenty wrong with NCLB and blunt-force accountability.  But if it disappeared tomorrow, Danny and Olivia would not suddenly be kickin’ it on the playground.  Well, maybe for 1o minutes after piano lessons and before the gourmet cooking class…

Farms, Field Trips and Test Scores

Published by Robert Pondiscio on October 26, 2009 in Assessment and Testing and Curriculum. 6 Comments

The New York Times rode along with 75 Harlem kindergarteners last week on a field trip to the Queens County Farm Museum to  gaze at cows and sheep “not only for a glimpse of rural life, but to rack up extra points on standardized tests.”

New York State’s English and math exams include several questions each year about livestock, crops and the other staples of the rural experience that some educators say flummox city children, whose knowledge of nature might begin and end at Central Park. On the state English test this year, for instance, third graders were asked questions relating to chickens and eggs. In math, they had to count sheep and horses.

The Harlem Success Academy has “invented a form of test preparation,” in the Times’ telling. “The schools haul their students to a farm each year, hoping to expose them to the rural life and lift their scores.” 

Someone here may be doing a teeny bit of overselling.  If HSA has taken to heart the connection between their students’ background knowledge and reading comprehension, that’s terrific.   Broad general knowledge certainly correlates with reading ability, but the test of a school’s dedication to that proposition is best measured in its commitment to a rich, well-rounded curriculum day after day, not the occasional field trip.  Unfortunately, the Times story doesn’t shed any light on the school’s overall approach to building background knowledge apart from its ostensibly novel “field study” idea.

Mind you, I’m thrilled to see the Times point out that “prior knowledge of a subject can significantly improve a child’s performance on tests.” It’s a connection that can’t be made too often. It might have been more helpful however, had they substituted “reading comprehension” for “performance on tests” in that sentence.   Creating the impression that kids should see cow or pick a pumpkin because farming might come up on a test years later strikes me as a bit of a stretch (whether it’s on the part of the Times or the school is unclear).   Background knowledge and vocabulary move in mysterious ways, creating unexpected and unpredictable connections.  At the Early Ed Watch Blog, Lisa Guernsey offers a somewhat more nuanced take:

A child who has explored a pumpkin patch will have a much easier time in the future when he or she comes across paragraphs about vines and tendrils, maturing fruit and harvest time. And it’s not just children’s reading skills, of course, that can improve. Their grasp of science and social studies becomes more sophisticated too.

Indeed, if there’s anything that rankles about the Times account, it’s viewing a field trip through the simple—and simplistic—lens of testing.  “I want to do better on homework and tests,” five-year-old Julliana Jimenez tells the paper.  At the risk of being retrograde, it’s a bit dispiriting to hear a kindergartener expressing any concern at all about tests, which don’t start until 3rd grade in New York.  One wonders where she picked it up.  Build broad general knowledge in children.  That will lead to broad language competence.  Let the testing take care of itself.

“Reverse Engineering Academic Upbringing”

Published by Robert Pondiscio on October 19, 2009 in Assessment and Testing, Higher Education and accountability. 1 Comment

The University of Nevada, Las Vegas is launching an ambitious research project to figure out why so many of its freshmen need remediation in reading and math.  Every incoming student will be evaluated “to reverse-engineer his academic upbringing,” UNLV president Neal Smatresk tells the Las Vegas Sun.  Since eighty percent of UNLV’s undergrads come from a single source, the state’s own Clark County School District, Smatresk hopes to gain particularly vivid insights.

Data gathered from the academic assessments would be shared with school districts and could help educators identify and correct patterns of weakness, whether it be general flaws in teaching philosophies or student study habits.  Clark County Schools Superintendent Walt Rulffes said the research findings could offer important insight into the root causes of the problems requiring remediation.

“The possibility that the district will be able to identify clusters of underachieving students, and trace them to not only individual campuses but individual classrooms, has Clark County’s teachers union on edge,” the paper notes. 

Last year, more than a third of Nevada’s high school graduates who enrolled at the state’s universities and colleges required remedial classes in English and mathematics, at a cost of over $2 million.

Do NAEP Scores Have Legs at the Polls?

Published by Robert Pondiscio on October 15, 2009 in Assessment and Testing, Education News and accountability. 1 Comment

In New York, 80 percent of 8th graders met the state’s standards in math this year, up from 59 percent two years ago.  But the National Assessment of Educational Progress (NAEP) results released yesterday paint a different picture.  Only 34 percent of the state’s 8th graders are considered proficient, a modest increase from 2007 levels.  NAEP scores for the Empire State’s 4th graders actually declined, while the percent passing the state’s own test went up.  This renewed charges that New York is making itself look good by lowering standards.   Diane Ravitch puts it plainly: ”The fabulous ‘gains’ reported last spring, we now know, were based on dumbed-down tests and dubious scoring of the tests in Albany,” she writes in today’s New York Post.

On the one hand, there is nothing new here, and New York is not alone in this boat.  Disconnects between the results for NAEP and state tests have been well known and much discussed for years.  The open question is whether state tests have now been sufficiently discredited in the minds of voters to make them a political liability.  The even larger question is whether the failure of test-driven accountability to move the needle will feed voter resentment, turning testing into a legitimate campaign issue in state and local races this November and beyond.

“There’s a palpable backlash against testing across much of the great American middle class,” Fordham’s Checker Finn recently observed.  ”We need to face the fact that testing, particularly high-stakes uses of test results for students and teachers alike, are deeply unpopular outside policymaker circles and could well lose rather than gain political traction in coming years.” 

The first test case may come in New York City, where Mayor Michael Bloomberg is up for a third term and campaigning on his education record. The New York Times points out this morning how the NAEP results are ”sharply contradicting the results of state-administered tests that showed record gains.”  NAEP results for the city itself will not be available for several weeks, but Bloomberg’s opponent, Democrat Bill Thompson, is attempting to make political hay nonetheless.  A spokesperson quoted in the New York Times today calls the Bloomberg administration the “Madoff of the American education system” and a “national disgrace.”  Bloomberg has a commanding lead in the polls, but his opponent is clearly trying to turn the Mayor’s record on education, a perceived strength, into a liability.  Will it play?  A Marist poll last Spring showed New York voters approved of the Mayor’s handling of the schools by a 51-to-41 percent margin.  It bears watching to what degree, if any, the testing issue moves those numbers.

No Child Left Behind, it has been widely observed, is a “tainted brand.”  But is “accountability” still a winner at the polls?  What NAEP seems to be telling us is that we’ve had a whole lot of test-driven accountability (and a whole lot of education spending) without a whole lot of results.  That said, it’s not an easy issue for voters to wrap their heads around.  I suspect it will be easier and more efficacious to get voters cranky about their kids education being reduced to a joyless grind.  “Prep and test schooling” does not roll as trippingly off the tongue as “tax and spend liberal” but it probably resonates more with voters than trying to explain cut scores.

Flatline! Call a Code Blue!

Published by Robert Pondiscio on October 14, 2009 in Assessment and Testing. 4 Comments

Reactions to today’s dispiriting NAEP scores….

“The trend is flat; it’s a plateau. Scores are not going anywhere, at least nowhere important.  That means that eight years after enactment of No Child Left Behind, the problems it set out to solve are not being solved, and now we’re five years from the deadline and we’re still far, far from the goal.” (Chester E. Finn, Jr. Thomas B. Fordham Institute)

“Had we had 19 years of flat results and one year of increases in one subject, we wouldn’t celebrate. Similarly, we shouldn’t press the panic button over one year of stalled growth in one subject…this is far from convincing evidence that NCLB failed or education reform is doomed.” (Andy Smarick @ Flypaper)

“It’s clear from the data at both grade levels that we still have a long way to go to effectively prepare all of our elementary and middle school students for the world that awaits them in high school and beyond.” (Kati Haycock, President of The Education Trust)

“Supporters of the No Child Left Behind Act–and I’ve generally been one of them–hoped that the law would catalyze a major upward move in student achievement. That hasn’t happened.” (Kevin Carey @ The Quick and The Ed)

“Seeing stuff flat-line is not what we want as a country — seeing achievement gaps that are unacceptably large.  The status quo isn’t good enough. We have to get dramatically better.”  (Secretary of Education Arne Duncan)

“We’re losing ground to our international competitors every year.  It’s a situation that calls for dramatic improvement. Unfortunately there seems to be apathy across the country.” (David P. Driscoll, chairman of the National Assessment Governing Board)

“The current system is producing school teachers who do not have a strong background in math themselves and may even be ‘afraid’ to teach math to pre-K students…if we want to improve students’ proficiency in math, we have to improve teachers‘ proficiency too. (Lisa Guernsey @ Early Ed Watch)

NAEP Math Scores Flat for 4th Graders; Up in 8th

Published by Robert Pondiscio on October 14, 2009 in Assessment and Testing. 1 Comment

New NAEP scores are out this morning:  No increase for 4th graders from 2007 to 2009; 8th graders are up two points.  From the IES release:

For the first time since the assessment began, 4th graders showed no overall increase at the national level, although they scored significantly higher in 2009 than when the assessment began in 1990.  For 8th graders, scores in 2009  were higher when compared to both 2007 and 1990.  These nationwide patterns also held for most student subgroups.

The report is here.  EdWeek’s Sean Cavanagh is first out of the box with analysis here.

Grading the Common Core Standards

Published by Robert Pondiscio on October 8, 2009 in Assessment and Testing, Curriculum and Education News. 5 Comments

A new report from the Fordham Foundation gives a grade of “B” to the draft of the proposed “Common Core” standards in ELA and Math.

Fordham’s report, Stars by Which to Navigate: Scanning National and International Standards in 2009, asked subject-matter experts to review the “content, rigor, and clarity of the first public drafts of the ‘Common Core’ standards” as well as the reading, writing and mathematics frameworks of NAEP, TIMSS, and PISA.  How’d they do?

Common Core Reading/Writing/Speaking & Listening: B
Common Core Math: B
NAEP Reading/Writing: B
NAEP Math: C
TIMSS Math: A
PISA Reading: D
PISA Math: D

The executive summary (I have not read the full report, which was just released this morning) makes a couple of important points, explaining and justifying the “B” grade for the common standards:

The document properly acknowledges that essential communication skills must be embraced and addressed beyond the English classroom….These skill-centric standards do not, however, suffice to frame a complete English or language arts curriculum. Proper standards for English must also provide enough content guidance to help teachers instill not just useful skills, but also imagination, wonder, and a deep appreciation for our literary heritage. Despite their many virtues, these skills-based competencies cannot serve as a strong framework for the robust liberal arts curricula that will prepare young Americans to thrive as citizens in a free society. States adopting these standards must, therefore, be very careful about how they supplement them so as to achieve that goal.

 Hard to disagree with any of that, and the B grade sounds fair.  “The Common Core standards are off to a good start,” says Fordham’s Checker Finn, “though there’s room for improvement—and a sound English curriculum will require plenty more than the valuable skills set forth here.”

Gimme One Good Reason

Published by Robert Pondiscio on October 2, 2009 in Assessment and Testing and Educational Policy. 2 Comments

I’ve been as critical of the squishy, content-free proposed national ELA standards as anyone, but over at Flypaper Eric Ulas reminds us that there is at least one good reason to support national standards: an end to the, er, impressionistic definitions of reading proficiency from state to state.  Ulas assumes we would have a single national test (I do too) to accompany national standards.  This would mean apples to apples comparisons and presumably an end to the race to lower cut scores.  The sunshine that would result from a national test would go a long way toward a sensible conversation about what’s working, what’s not working and why. 

Also well worth your time is Tom Hoffman’s take on the standards at Tuttle SVC.  Tom and I don’t always agree, but he knows standards, and  his point that the proposed standards are “narrower, lower, and shallower than the Language Arts standards of high performing countries” is very persuasive and backed up with good examples.  “No country with high reading scores in international assessments conceives of the discipline of Language Arts as being limited to literacy skills, or “college- and career-readiness,” as the Common Standards do,” he writes.

Inferencing Test

Published by Robert Pondiscio on October 1, 2009 in Assessment and Testing. 22 Comments

“After failing to move a runner past first base for the entire game, the Giants sent Davis to the plate with the potential tying and winning runs in scoring position.  Unfortunately, he hit into a 6-4-3 double play to end the game.”

  1.  How many outs were there when Davis came to bat?
  2. To whom did he hit the ball?
  3. Describe the kind ball he hit (pop up? Line drive? etc.)
  4. What was the final score of the game
  5. How many runners were on base?

If you are able to answer all five of these questions (#5 is tricky) is it because you have mastered the ”reading skill” of making inferences. Or because your knowledge of baseball fills in the gaps for you?

Moving the Chains

Published by Robert Pondiscio on September 30, 2009 in Assessment and Testing. 3 Comments

Football fans see it time and again:  It’s 4th down and short yardage.  An official standing 30 or 40 feet away from the play sees a running back hurl himself full throttle into a forest of 300-pound linemen and disappear beneath a collapsing pile of players, a football buried somewhere against his body.   Chaos everywhere, yet the official, with unquestioned authority places the ball he lost sight of on the exact spot on the ground where forward momentum stopped and calls for the chains.  Play stops and the fans grow quiet as a team of officials runs in from the sidelines and takes a precise-to-the-inch measurement of the ball’s location.  If the any part of the ball is beyond the plane of the outstretched chain, a first down is awarded.  The crowd goes wild. 

FIRST DOWN by MIKECNY.

Never mind that the linesman is merely estimating the ball’s position.  Never mind that the ten-yard length of chain was placed based on an eyeball approximation of where the series of downs began three plays ago.  Never mind that every play in the series of downs begins and ends with a best guess (the wide receiver was knocked out of bounds at about the 35-yard line) When it’s time to determine whether or not a first down is to be awarded, football is suddenly a game of inches. 

Games, playoff hopes, bowl bids and careers turn on a guess–or a series of guesses.  But no one seems to question it.  Call for the chains!  If you stop and think about it, this doesn’t make a lot of sense.  The answer however is simple: Don’t think about it.

Here are a few more things not to think about:

  • Writing in the New York Times, Todd Farley, the author of the book “Making the Grades: My Misadventures in the Standardized Testing Industry” describes getting a part-time, $8 an hour job scoring fourth-grade, state-wide reading comprehension tests after a five-minute interview.  “Arbitrary decision is the rule, not the exception,” he writes.  
  • “Bowen Elementary was part of what [Washington, DC] officials hailed as the success story of their 2008 standardized test results,” reports the Washington Post.  “But Bowen also had four classrooms where children erased wrong answers and replaced them with correct ones at abnormally high rates.”  The paper reports there were elevated numbers of erasures at six schools involving classrooms with 573 students.  CTB McGraw-Hill declared the data “inconclusive,” and no teachers or administrators have been accused of wrongdoing, the Post reports.  
  • In New York State seventh graders who answered just 44 percent of questions correctly on the state math test were given a passing grade. “Three years ago, the threshold for passing was 60 percent,” the New York Times reports. “In fact, students in every grade this year could slide by with fewer correct answers on the math test than in 2006.”
  • Teacher Diana Senechal recently described an experiment in which she was able to “pass” several standardized tests just by guessing and without even looking at the tests. 
  • “Policy makers define good education as higher test scores,” writes Diane Ravitch. “But students can get higher scores in reading and mathematics yet remain completely ignorant of science, the arts, civics, history, literature and foreign languages.” 

We know this.  We see it all around us, but like the football fan caught up in the arbitrary kabuki dance of the moving of the chains, we accept it, applaud it or moan about lousy spots, but the game goes on. 

“There must be a better way,” Pat Summerall, an N.F.L. veteran and broadcaster said in a recent New York Times article. “Because games are decided, careers are decided, on those measurements.”  He was talking about measuring for first downs.  “There’s a certain amount of drama that is involved with the chains,” said New York Giants president, John Mara in the same article. “Yes, it is subject to human error, just like anything else is. But I think it’s one of the traditions that we have in the game, and I don’t think any of us have felt a real compelling need to make a change.”

“With national standards will come national standardized tests, so it’s an especially good time to rethink how these exams are scored, and by whom,” Dana Goldstein sensibly observes at The American Prospect’s Tapped blog.  “Perhaps teachers and principals should be scoring tests, not $8 an hour part-timers. In that case it would be important, especially with the push for merit pay, to make sure teachers aren’t grading their own students’ tests, to decrease the temptation to engage in foul play.”

Like the theatrical measurement of a first down in football, we want to rely on precise measurements of an imprecise process to make high stakes decisions on everything from federal funding to merit pay to whether a teacher keeps his or her job at all.  “I understand that tests are far from perfect and that it is unfair to reduce the complex, nuanced work of teaching to a simple multiple choice exam,” Education Secretary Arne Duncan recently observed. 

Right.  It’s way more complicated than that. 

But it’s 4th down!  Call for the chains!  Take a measurement.  How else are we going to know?