Choosing Curriculum Without Evidence

by Robert Pondiscio
April 13th, 2012

If you wanted to improve medical care, would you focus on hospital administration and patient insurance?  Or would you look at the treatment doctors were giving patients?  Would you try to improve a sports team’s won-loss record by focusing on stadium layout and the team’s travel schedule?  Then why, ask Brookings’ Matthew Chingos and Russ Whitehurst, do education policy makers focus most of their attention on academic standards, teacher evaluation, and school accountability policies?  Shouldn’t we be looking instead at instructional materials?

“There is strong evidence that the choice of instructional materials has large effects on student learning—effects that rival in size those that are associated with differences in teacher effectiveness,” the two write in a new paper from Brookings, Choosing Blindly: Instructional Materials, Teacher Effectiveness, and the Common Core.

“Whereas improving teacher quality through changes in the preparation and professional development of teachers and the human resources policies surrounding their employment is challenging, expensive, and time-consuming, making better choices among available instructional materials should be relatively easy, inexpensive, and quick.”

There’s one big hurdle to clear in correcting this rather obvious problem: Little effort has been made by the field to differentiate effective curricular materials from ineffective ones.  In fact, in most states, districts and schools, it’s nearly impossible to know what materials are being used at all.

“In every state except one, it is impossible to find out what materials districts are currently using without contacting the districts one at a time to ask them. And the districts may not even know what materials they use if adoption decisions are made by individual schools. The National Center for Education Statistics (NCES), which has the mission of collecting and disseminating information related to education in the U.S., collects no information on the usage of particular instructional materials.”

Chingos and Whitehurst predict the blindness on curriculum will become a critical problem for Common Core Standards implementation.  “Publishers of instructional materials are lining up to declare the alignment of their materials with the Common Core standards using the most superficial of definitions,” they note.  “The Common Core standards will only have a chance of raising student achievement if they are implemented with high-quality materials, but there is currently no basis to measure the quality of materials. Efforts to improve teacher effectiveness will also fall short if they focus solely on the selection and retention of teachers and ignore the instructional tools that teachers are given to practice their craft.”

The paper offers up a number of suggestions:  State education agencies should collect data from districts on the instructional materials in use in their schools.   also wants to see the National Governors Association (NGA) and Council of Chief State School Officers (CCSSO) “put their considerable weight behind the effort to improve the collection of information on instructional materials in order to create an environment in which states, districts, and schools will be able to choose the materials most likely to help students master the content laid out in the Common Core standards.”

Chingos and Whitehurst are dead-on in their critique of ed reform’s indifference to curriculum and materials.  When we focus on the mechanism by which schools are  created, managed, financed or evaluated, we are assuming that what kids learn, and with which materials, is pretty much settled, or doesn’t really matter.  All that’s left to do is figure out what works in terms of delivery of instruction and grow it, or figure out what doesn’t work and shut it down.  Any teacher who has worked with different literacy or math programs can easily attest this is not the case.

Worse Than Awful: An Insider’s View of Educational Publishing

by Robert Pondiscio
February 23rd, 2012

Can’t figure out a problem in your child’s math textbook?  Maybe it’s not you. “It could be that key information or steps are missing, that the problem involves a concept to which your child hasn’t yet been introduced.  “Perhaps the problem is structurally unsound for a host of other reasons,” notes veteran textbook writer and editor Annie Keeghan at the blog Open Salon.

The “new normal” in educational publishing is “a severe lack of oversight in the quality of curriculum being produced” and a “frightening apathy” to do anything about it.  Keeghan’s piece, “Afraid of Your Child’s Math Textbook? You Should Be” is a jeremiad.  It does for textbook publishing what The Jungle did for the meatpacking industry.

Keeghan paints a bleak and dispiriting picture of a business gutted by mergers, competition for fewer available dollars, and an increased focus on sales and marketing at the expense of producing quality products.  Materials rushed to market at breakneck speed are “inherently, tragically flawed.”  Plus the pool of qualified writers and editors is drying up, and those doing the work “often don’t have the necessary skills or experience to produce a text worthy of the publisher’s marketing claims,” she writes.

Otto von Bismarck famously quipped that laws and sausages are two things you should never watch being made.  What might he have said about textbooks?

“Here’s how it works: Many publishers solicit developers, often on the Internet and from all over the world, looking for the best bid on a project. With competition this fierce, developers are forced to drastically lower their rates just to stay in business (and publishers exploit this fact). Let’s say a publisher hires a developer for a certain low-bid fee to produce seven supplemental math books for grades 3-8. The product specs call for each student book and teacher guide to have page counts of roughly 100 pages and 80 pages, respectively. The publisher wants these seven books ready for press in five weeks—over 1,400 pages. To put this in perspective, in the not too recent past at least six months would be allotted for a project of this size. But publishers customarily shrink their deadlines to get a jump on the competition, especially in today’s math market. Unreasonable turnaround times are part of the new normal, something that almost guarantees a lack of quality right out of the gate.”

Keeghan has stopped writing educational books, finding there’s no longer any satisfaction in the work, no demand for a good product, or even a way to make a decent living at it.  These days, she says she only accepts copyediting work.  And that’s bad enough.

“When I’m hired to copyedit, the profound errors I see in content are often staggering enough that grammar and punctuation seem immaterial. Sometimes the content in the student materials is so poor—steps omitted, unclear directions, concepts introduced when they’re not developed till later in the text—that it boggles the mind it got past a content editor. With so many errors rampant at this stage of editing, rewriting is hastily done and it’s only inevitable that some errors will show up in the final printed product. And with a different copyeditor on each book, there are those who don’t even think about, or have the experience to recognize, the content issues so they go unaddressed.”

When she points out profound problems with educational materials, Keeghan writes, a typical response is, “The publisher knows it’s bad. Just do the best you can.” The losers, naturally, are students, who are caught in the squeeze between a poorly executed product and a marketing push to maximize profits.“One must conclude that students and their education, if this is judged against product quality, is becoming an increasingly low priority,” Keeghan writes.

“And so, I say to parents: Take a good look at the materials your children are bringing home. And to educators: Look at what you’re purchasing. Don’t be satisfied with the classic “thumb through” and don’t take those marketing materials or the sales pitch at face value. Take the time to study the materials; match them to your state’s desired standards and preferred benchmarks. If they’re not a good fit, take a pass and develop your own if you must. The only way kids are going to become better educated through the materials you buy, to increase their rankings among those 30 other countries, is to break the cycle and stop buying those books that are—there’s no other way to put it—crap.”

Stunning.  Sobering.  And even more so if the reader comments following the piece are credible.  Several are from publishing industry types largely confirming Keeghan’s bleak assessment.

A Little More Text, A Little Less Self

by Robert Pondiscio
December 19th, 2011

When studying a story or an essay, is it possible to be too concerned with what the author is saying? In an opinion piece in Education Week, Maja Wilson and Thomas Newkirk complain the publisher’s criteria for Common Core State Standards are overly “text dependent,” discouraging students from bringing their own knowledge and opinions to bear on their reading.

Wilson, a former high school English teacher, and Newkirk, a University of New Hampshire English professor applaud the guidelines’ “focus on deep sustained reading—and rereading.” However they pronounce themselves “distressed” by the insistence that students should focus on the “text itself.”

“There is a distrust of reader response in this view; while the personal connections and judgments of the reader may enter in later, they should do so only after students demonstrate ‘a clear understanding of what they read.’ Publishers are enjoined to pose ‘text-dependent questions [that] can only be answered by careful scrutiny of the text … and do not require information or evidence from outside the text or texts.’ In case there is any question about how much focus on the text is enough, ‘80 to 90 percent of the Reading Standards in each grade require text-dependent analysis; accordingly, aligned curriculum materials should have a similar percentage of text-dependent questions.”

Consider me undistressed. If this means less reliance on the creaky crutch that is “reader response” in ELA classrooms, then I’m very nearly overjoyed.

The very worst that can be said about an over-reliance on text-dependent questions is that it’s an overdue market correction. As any teacher can tell you, it’s quite easy to glom on to an inconsequential moment in a text and produce reams of empty “text-to-self” meandering using the text as nothing more than a jumping off point for a personal narrative. The skill, common to most state standards, of “producing a personal response to literature” does little to demonstrate a student’s ability to read with clarity, depth and comprehension.

Indeed, educator, author and occasional Core Knowledge Blog contributor Katharine Beals points out in a response to the piece that Wilson and Newkirk have it precisely backwards: research from cognitive science suggests that making external associations during reading can actually worsen comprehension. She cites a paper by Courtenay Frazier Norbury and Dorothy Bishop which found that “poor readers drew inferences that were distorted by associations from their personal lives. For example, when asked, in reference to a scene at the seashore with a clock on a pier, ‘Where is the clock?’ many children replied, ‘In her bedroom.’”

“Norbury and Bishop propose that these errors may arise when the child fails to suppress stereotypical information about clock locations based on his/her own experience. As Norbury and Bishop explain it: ‘As we listen to a story, we are constantly making associations beween what we hear and our experiences in the world. When we hear “clock,” representations of different clocks may be activated, including alarm clocks. If the irrelevant representation is not quickly suppressed, individuals may not take in the information presented in the story about the clock being on the pier. They would therefore not update the mental representation of the story to include references to the seaside which would in turn lead to further comprehension errors.’

Struggling readers in particular would benefit from a lot more text and a lot less self. As Beals explains, “Text-to-self connections, in other words, may be the default reading mode (emphasis mine) and not something that needs to be taught. What needs to be taught instead, at least where poor readers are concerned, is how not to make text-to-self connections.”

Wilson and Newkirk illustrate their concern about over-reliance on text by describing their preferred way of teaching Nicholas Carr’s 2008 essay from The Atlantic, “Is Google Making Us Stupid?”

“Before assigning the essay, we would have students log their media use for a day (texts, emails, video games, TV, reading, surfing the Internet) and share this 24-hour profile with classmates. We might ask students to free-write and perhaps debate the question: “What advantages or disadvantages do you see in this pattern of media use?” This ‘gateway’ activity would prepare students to think about Carr’s argument. As they read, they’d be mentally comparing their own position with Carr’s. Surely, we want them to understand Carr’s argument, but we’d help them do that by making use of their experiences and opinions.”

It’s critical to understand that this approach to teaching Carr’s essay would not be verboten under CCSS publishing guidelines, which have nothing whatsoever to say about teaching methods. In fact, there’s much to recommend Wilson and Newkirk’s approach. But the test of whether the students understand Carr’s line of argument has nothing to do with the “gateway” activity, which serves mostly as an engaging hook to draw students into Carr’s thesis. Students cannot be said to have understood the piece—or any piece—of writing without the ability to show internal evidence.

Thus if publishers are “enjoined to pose text-dependent questions [that] can only be answered by careful scrutiny of the text” that is at heart not a teaching question–it’s an assessment question that probes whether or not the student understands the text.

All those connections—to our own experience, to other works of literature, make the study of literature thrilling and rewarding. But for those connections to be deep and meaningful requires more than just the superficial, paper-thin connections that too often pass for “personal response.”

What often gets lost in our rush to engage young readers and make their reading personally relevant is the simple fact that text has communicative value. When someone commits words to print, they mean to communicate facts, ideas, imagery or opinions. They should expect, if they’ve done their job well, to be understood. Might the reader have a response? Let’s hope so. But unless they have understood the author’s words and intent clearly, any response they make is less than satisfying and may not be particularly relevant as a “response.”

The bottom line: Demonstrating comprehension based on what a text says is not a problem. It’s a baseline skill for any literate human being.

The Texas Textbook “Urban Myth”

by Robert Pondiscio
March 29th, 2010

A California state legislator is promising to draft a bill to stop controversial changes in Texas’ textbooks from reaching students in the Golden State.  But the Los Angeles Times cautions its readers to consider how California injects politics into textbook rules before looking down their noses at Texas.

“The state regulates the portrayal of genders, minority groups, the elderly and the disabled by requiring proportional representation that also cannot show any group in a negative light. Thus, as education expert Diane Ravitch writes, the elderly must be portrayed as fit and lively even if reality tells us that some cope with illness and disability. Publishers have been discouraged from portraying people in poor countries as poor — because that would stereotype those nations — and told to soften language on AIDS in Africa so as not to reflect badly on that continent.”

A 2006 bill, which was stopped by a threatened veto by California’s governor Schwarzenegger, would have required textbooks “to show a diversity of sexual orientation and would have led to basing choices of important people to include in textbooks ”not solely on their accomplishments but on their sexual orientation,” notes the paper, which complains that “each textbook adoption in the state becomes a battle in which religious, ethnic and other groups demand changes so that they are seen in a more positive light.

“Textbook information should be compiled by scholars and written by talented storytellers, not by politicians, interest groups or publishers’ corporate committees,” concludes the Times, which says statewide textbook adoptions are part of the problem. “If school districts were empowered to pick their own books, allowing a more competitive textbook market to thrive, there would be less opportunity for lobbyists of any stripe to have undue influence over the process.”

That’s pretty much what already happens, reports the Texas Tribune, a nonpartisan public media organization, which says the as-Texas-goes-so-goes the-nation’s-textbook idea is something of an urban myth.  “I’ve been in this job about three and a half years, and I see it reported all the time,” Jay Diskey, the executive director of the Association of American Publishers. “I give my explanation to reporters, and about half of them believe me and half of them don’t.”  Publishers start with a “core national narrative and edit to suit the sensitivities and curriculum standards of various states and districts,” notes the Tribune, which also points out an irony in the Texas textbook flap: “The more the state board makes a political circus out of the process, the less likely any of its ideology will seep into books for other states.”

Finally, Newsweek’s David A. Graham gives several reasons why concerns over textbooks in Texas are overblown.  It’s not that bad, he argues, and it’s Texas’ business. Besides, he concludes, “it’s not like high-schoolers pay any attention to their textbooks anyway.”

“It Boggles My Mind the Kind of Power We Have”

by Robert Pondiscio
February 1st, 2010

The article has been out for nearly a month, but I just caught up to “Revisionaries,” Mariah Blake’s exceptional piece on the curriculum  battles in Texas in the current issue of Washington Monthly.  It’s conventional wisdom that Texas wields outsize influence on textbooks nationwide because of its statewide adoption policies.  With California, the other textbook behemoth, putting off buying new books until 2014, Texas now has “unparalleled power to shape the textbooks that children around the country read for years to come,” Blake writes.  That power largely rests, she says, with Don McLeroy.

The jovial creationist sits on the Texas State Board of Education, where he is one of the leaders of an activist bloc that holds enormous sway over the body’s decisions. As the state goes through the once-in-a-decade process of rewriting the standards for its textbooks, the faction is using its clout to infuse them with ultraconservative ideals. Among other things, they aim to rehabilitate Joseph McCarthy, bring global-warming denial into science class, and downplay the contributions of the civil rights movement.

Blake’s article is a fascinating trip through the last 50 years or so of Texas politics and conservative activism, most notably the discovery in the 1960s by Norma and Mel Gabler, a housewife and an oil-company clerk, that Texas had “a little-known citizen-review process that allowed the public to weigh in on textbook content.” 

When textbook adoptions rolled around, the Gablers would descend on school board meetings with long lists of proposed changes—at one point their aggregate “scroll of shame” was fifty-four feet long. They also began stirring up other social conservatives, and eventually came to wield breathtaking influence. By the 1980s, the board was demanding that publishers make hundreds of the Gablers’ changes each cycle. These ranged from rewriting entire passages to simple fixes, such as pulling the New Deal from a timeline of significant historical events (the Gablers thought it smacked of socialism) and describing the Reagan administration’s 1983 military intervention in Grenada as a “rescue” rather than an “invasion.”

To avoid running afoul of the Gablers and other activists, “many publishers started self-censoring or allowing the couple to weigh in on textbooks in advance,” Blake notes. 

McLeroy describes his current efforts, apparently in earnest, as a never-ending battle for truth, justice, and the American way.  “There are people out there who want to replace truth with political correctness. Instead of the American way they want multiculturalism. We plan to fight back—and, when it comes to textbooks, we have the power to do it,”  he tells Blake, concluding with stunning candor:  “Sometimes it boggles my mind the kind of power we have.”

Taking the Bat Out of The BOE’s Hands in Texas

by Robert Pondiscio
April 14th, 2009

Texas state legislators have apparently had enough of the endless arguments over evolution and other charged topics that regularly put the state’s Board of Education in the national spotlight.  The Wall Street Journal reports they are considering stripping the Board of its authority to set curricula and approve textbooks. 

While the science standards have drawn the most attention, the 15-member elected board has been embroiled in other controversies as well. Last year, it rejected a reading curriculum that teachers had spent nearly three years drafting. In its place, the board approved a document that a few members hastily assembled just hours before the vote.

Various proposals being drawn up in Texas would transfer curriculum oversight and textbook adoptions to the state education agency, a legislative board or the commissioner of education. “Other bills would transform the board to an appointed rather than elected body, require Webcasting of meetings, and take away the board’s control of a vast pot of school funding,” the Journal reports.

Evolution “Strengths and Weaknesses” Voted Down in Texas

by Robert Pondiscio
January 23rd, 2009

The Texas Board of Education voted Thursday to drop a 20-year old state requirement that high school science teachers cover “strengths and weaknesses” in the theory of evolution. The vote is being characterized as a major defeat for social conservatives and sharply divided the Board.

“Under the science curriculum standards recommended by a panel of science educators and tentatively adopted by the board, biology teachers and biology textbooks would no longer have to cover the ‘strengths and weaknesses’ of Charles Darwin’s theory that man evolved from lower forms of life,” the Dallas Morning News reports.

A panel of science teachers had recommended that the “strengths and weaknesses” language be dropped.  Critics had argued that the word weaknesses “has become a code word in the culture wars to attack evolution and promote creationism.”  The Texas science standards have ripple effects from coast-to-cost, influencing how textbook publishers publishers handle the topic, since the Lone Star state is the largest statewide textbook adoption state. 

Critical Math

by Robert Pondiscio
March 12th, 2008

Proponents of a more traditional, rigorous approach to teaching mathematics should read this piece from the Los Angeles Times about the success a struggling Hollywood elementary school has enjoyed with Singapore Math.

Several Core Knowledge schools have reported strong results from Singapore and Saxon math programs, and the paper does a good job of showing why. Describing what appears to be a standard timed drill (the dreaded “drill and kill” that reform advocates blithely dismiss) the Times smartly reports: “What isn’t obvious to a casual observer is that this drill is carefully thought out to reinforce patterns of mathematical thinking that carry through the curriculum. ‘These are ‘procedures with connections,’ math coach Robin Ramos said, arranged to convey sometimes subtle points. This thoughtfulness — some say brilliance — is the true hallmark of the Singapore books, advocates say.”

As the paper notes, California recently became the first state to include the Singapore series on its list of state-approved elementary math texts, and will subsidize schools’ purchase of the books. “Being on the list puts an important imprimatur on the books,” notes the Times, “because California is by far the largest, most influential textbook buyer in the country.”

Also this week: an anticipated report from the National Mathematics Advisory Panel, which is expected to urge U.S. teachers to promote “quick and effortless” recall of arithmetic facts in early grades. Taken together, it’s a potent one-two punch that coupled with a rising tide of parent activism, may be turning the tide against reform or constructivist math programs like Everyday Math.

A consummation devoutly to be wished.