A Wince a Day Keeps My Hopes at Bay

by Lisa Hansel
April 17th, 2013

Tags: , ,
Posted in Education Theory, Literacy, Research and Reports | 12 Comments »

I’ve long been aware of the widespread misconception that comprehension, critical thinking, and the like are content-free skills. Wanting to help correct that delusion is one of the main reasons I joined the Core Knowledge Foundation.

Having been with the foundation for a little over a month, I’m seeing the skills-don’t-need-content fallacy everywhere. My neck is starting to ache from all this wincing.

Today’s encounter really caught me by surprise. It came from one of my favorite organizations: the American Library Association (ALA). Maureen Sullivan, ALA’s president, wrote a compelling plea on the Huffington Post to save the nation’s school libraries:

Recently the ALA has tracked multiple news reports regarding school districts that have placed school librarian positions on the chopping block in response to budget deficits…. For example, Pasco County (Fla.) School Superintendent Kurt Browning proposed a plan to eliminate 28 school media specialist positions in the next school year because of a budget shortfall…. In Sarasota, Florida, more than 18,000 middle and high school students may be without a school librarian. Local school board officials there are considering a proposal to eliminate all school media specialists…. School Districts in Louisiana, Maryland, Washington State and New York State also are considering proposals or reorganization plans that would eliminate school librarians.

We all know that there are far too many students without books in their homes and without the safe streets or bus fare necessary to access a community library. School libraries are essential—the very fact that Sullivan has to plead for them is a sad commentary on America’s priorities.

All of that is wince worthy. But this is what got me:

School librarians help more than 30 million students each week navigate a vast landscape of digital content, because the majority of students still lack the ability to analyze information found online.

The Pew Research Center’s Internet & American Life Project Online Survey of Teachers found that although the Internet has opened up a vast world of information for today’s students, their digital literacy skills have yet to catch up. Twenty-four percent of those surveyed stated that students lack the ability to assess the quality and accuracy of information they find online. Another 33 percent reported that students lack the ability to recognize bias in online content.

Of course they “still” lack those abilities. Assessing the quality, accuracy, and potential biases of information—no matter where that information is found—can only be done by those with lots of content knowledge. By the very nature of schooling, students are almost always studying content that is new to them, so they very rarely have the extensive knowledge needed to make such judgments.

These questions are asked regarding information found online because adults want students to be able to use the internet more effectively. We might be able to teach students to be generally cautious and skeptical online, but for real analysis, content knowledge is the only option.

To make my point, I’m going to share two “mere facts” that will make us all wince. Fact 1: It’s not just the internet that is full of inaccurate information, even widely used mathematics textbooks are highly error-prone. Fact 2: Very few of us, even few our mathematics education professors, have “the ability to assess the quality and accuracy of information they find” in these textbooks.

The extensive errors in five widely used algebra textbooks were documented in chapter 3 of the report by the National Mathematics Advisory Panel. The lack of awareness of these problems has been explored by Hung-Hsi Wu, an emeritus mathematics professor at Berkeley. He places blame not on the math education professors (and certainly not on teachers), but on mathematicians:

As a mathematician surveying this catastrophic education mess, I have to admit that, when all is said and done, the mathematics community has to take the bulk of the blame. We think school mathematics is too trivial, and we think the politics of education is a bottomless pit not worthy of our attention. So we take the easy way out by ignoring all the goings-on in the schools…. even though we are daily confronted with evidence that it is not working.

Why doesn’t Wu blame the math education professors or the teachers—and why does he blame the mathematicians? Because he knows that this analysis of the accuracy of mathematics textbooks could only be done by those with deep knowledge of mathematics. Knowledge that, largely due to their neglect of the rest of us, only mathematicians have.

In school, when students are learning about things for the first time, why should we expect them to be able to analyze the information they find online? I can show you a 12-year-old boy who, having been crazy about dinosaurs since he first chewed on a T. rex, can analyze the accuracy of almost anything about dinosaurs. But that same boy would likely fall for the tree octopus.

Out of curiosity, I dug up the Pew survey Sullivan mentioned. The survey sample is not representative of all teachers; it is about two-thirds Advanced Placement teachers and one-third middle and high school National Writing Project teachers. Asked to rate their students excellent, very good, good, fair, or poor, the teachers rated

  • 61% fair or poor on “Ability to assess the quality and accuracy of information they find online.”
  • 71% fair or poor on “Ability to recognize bias in online content.”
  • 59% fair or poor on “Ability to use multiple sources to effectively support an argument.”

Are these results good or bad? We have no way of knowing. These teachers could be challenging their students with a steady stream of new information and ideas. Students may be acquiring broad knowledge that can provide a foundation for future studies. The fact that so many do not yet have the deep knowledge needed for independent online research need not be a great concern—it merely tells us that they need to learn more. Or, these students could be generally uninformed; expected to build analysis skills but not taught relevant knowledge, they may be headed for failure in future studies.

If Pew wanted to find out, it could do a follow-up study to investigate the students’ academic content knowledge. It would likely find, as so many cognitive scientists already have, that students’ analytical skills and content knowledge develop together.

 

How Two Poems Helped Launch a School Reform Movement

by E. D. Hirsch, Jr.
April 5th, 2013

Tags: , , ,
Posted in Core Knowledge, Education Theory, Research and Reports | 3 Comments »

This essay was published on The Atlantic’s website on March 29, 2013; it is reposted here with permission.

Right now, roughly 1,000 schools—public, private, rural, urban, and suburban—are implementing a curriculum plan called the Core Knowledge Sequence. That number is slated to increase significantly in the fall: Under the new Common Core State Standards, the state of New York is recommending the Core Knowledge Language Arts program for preschool through second grade.

It won’t be long before the Core Knowledge program will have helped educate more than a million children—an estimate that doesn’t count the several million children whose parents have taken them through Core Knowledge books such as What Your First-Grader Needs to Know. Judging from the evidence, this is a good thing. The Core Knowledge curriculum is based on the idea that students need actual knowledge, not just thinking skills, in order to succeed. As the program’s website explains:

It’s natural to assume that teaching lots of “stuff” isn’t important anymore when students can simply Google anything they need to know. But you probably take for granted how much “walking-around knowledge” you carry inside your head—and how much it helps you. If you have a rich base of background knowledge, it’s easier to learn more. And it’s much harder to read with comprehension, solve problems and think critically if you don’t.

As I turn 85, I find myself looking back on my own intellectual history with Core Knowledge. I’ve written four books on the theory behind all this activity. But the thought occurs: Perhaps sharing my personal epiphanies might be a good way of helping others understand the program’s character and scientific origins. More important, perhaps it would help mitigate two misconceptions: that reading is a technical skill and that Core Knowledge is impelled by reactionary nostalgia.

***

A crucial moment occurred about 60 years ago as I was in my first semester of teaching English to Yale freshmen. The poem under discussion that day was “Valediction Forbidding Mourning” by John Donne, and my interpretation was being challenged by a very sharp undergraduate.

The poem starts this way:

AS virtuous men pass mildly away,
     And whisper to their souls to go,
Whilst some of their sad friends do say,
     ”Now his breath goes,” and some say, “No.”
 
So let us melt, and make no noise,
     No tear-floods, nor sigh-tempests move;
‘Twere profanation of our joys
     To tell the laity our love.

The undergraduate insisted that it was a poem about death, since the poem forbids “mourning” and offers the image of a man dying quietly.

Most professors of English would agree that this is not a poem about dying. In Donne’s day, the word “mourning” did not have the limited, mortuary connotation it has now. True, the poet does say he is departing from his beloved, but he’s going on a real geographical trip. In the rest of the poem he explains that he’ll be coming back, and they will renew their love as before. The valediction is a “be seein’ ya,” not a “farewell.”

But nonetheless the poem can be read as a permanent farewell. In Donne’s famous image of a compass, the twin legs part from each other, then one leg takes a circular trip, but then the two legs come back together. All that could be read as a reuniting of two souls after death. There are other clues that make death a plausible interpretation—not just the word “mourning” in the title, but also the image of the dying man, and the poet’s insistence that he and his beloved are not like “dull sublunary lovers” who depend on each other’s physical presence. That could suggest some sort of posthumous spiritual reunion.

But my bright undergraduate didn’t even need to bring out those detailed arguments. He made a more decisive theoretical observation: He pointed out that then-current literary theory held that the intention of the poet is irrelevant. A poem goes out into the world as an artwork, a “verbal icon,” to be interpreted as readers wish, so long as their interpretations follow the public norms and conventions of language. That doctrine meant, said the undergraduate, that hisreading of the poem was just as valid as my reading, since both followed public norms and conventions. My immediate response was that his logic was absolutely right.

So, why was I teaching this class?

In 1954, Yale was the vibrant center of the “New Criticism” that had already begun to take over the teaching of literature in the high schools, mainly through the phenomenally successful textbook by Cleanth Brooks and Robert Penn Warren called Understanding Poetry. The theory was that you didn’t need to have a lot of biographical or historical information to understand poetry. You could learn to read any poem if you knew poetic conventions and techniques. The other influential text was The Verbal Icon by William K. Wimsatt, who, like Brooks, had been a professor of mine at Yale. All of them became dear friends despite our disagreements.

In those heady days when the Yale English department was rated tops in the nation, it had the feeling almost of a theological seminary for the new doctrines that freed the study of literature from its pedantic, historical trappings and treated works of literature intrinsically as literature—as “verbal icons.” Under this theory, the argument that my student made was right. His “reading” wasjust as valid as mine. Once he had mastered Understanding Poetry, why should I, or anyone, need to teach him how to read Donne’s poem?

***

Five years passed. I was now back from a Fulbright in Germany where I had completed my dissertation on William Wordsworth and Friedrich Schelling, and I was teaching at Yale again. I now thought I was ready to respond to the undergraduate’s challenge. I had explained in the introduction to my dissertation just why you do really need to know quite a lot of extrinsic things to understand even the simplest poem of Wordsworth.

When I was in Germany, I had eagerly read the works of humanistic theorists like Wilhelm Dilthey and philosophers like Edmund Husserl. I had also begun to read linguistics and cognitive psychology. I wrote up my musings as a 1960 article called “Objective Interpretation” in the Publications of the Modern Language Association. Besides citing a lot of eminent German theorists, I offered a concrete example: a simple Wordsworth poem along with two very different interpretations, one by Cleanth Brooks and the other by historical scholar F. W. Bateson. Here is the poem:

A SLUMBER did my spirit seal;
     I had no human fears:
She seem’d a thing that could not feel
     The touch of earthly years.
 
 No motion has she now, no force;
     She neither hears nor sees;
Roll’d round in earth’s diurnal course
     With rocks, and stones, and trees.

In Brooks’s view, the poem evokes a sense of futility—the lover’s “agonized shock” at watching his beloved turn into an inert object like a rock, stone, or tree:

Part of the effect, of course, resides in the fact that a dead lifelessness is suggested more sharply by an object’s being whirled about by something else than by an image of the object in repose. But there are other matters which are at work here: the sense of the girl’s falling back into the clutter of things, companioned by things chained like a tree to one particular spot, or by things completely inanimate like rocks and stones. … [She] is caught up helplessly into the empty whirl of the earth which measures and makes time. She is touched by and held by earthly time in its most powerful and horrible image.

In contrast, F. W. Bateson sees the poem building up to a sense of “pantheistic magnificence”:

The vague living-Lucy of this poem is opposed to the grander dead-Lucy who has become involved in the sublime processes of nature. We put the poem down satisfied, because its last two lines succeed in effecting a reconciliation between the two philosophies or social attitudes. Lucy is actually more alive now that she is dead, because she is now a part of the life of Nature, and not just a human “thing.”

As someone deeply immersed in Wordsworth, I could say authoritatively that Bateson caught the poet’s intended sense pretty well: He knew that nothing was really dead in Wordsworth’s nature. As the poet wrote in “The Prelude Book, III”:

To every natural form, rock, fruits, or flower,
Even the loose stones that cover the highway,
I gave a moral life: I saw them feel,
Or linked them to some feeling: the great mass
Lay bedded in a quickening soul, and all
That I beheld respired with inward meaning.

If Wordsworth had meant to imply the “dead, dead inertness” that Brooks found in the poem’s conclusion, he would hardly have ended the series “rocks and stones and trees.”

However, by favoring Bateson’s reading over Brooks’s, I was disobeying the New Critical doctrine that intention doesn’t matter. This raised a troubling contradiction. If there was no such thing as a “correct” interpretation, then a poem could mean one thing and its complete opposite. In other words, if the text was all you needed, you were led by a kind of Hegelian logic to the next dominant literary theory: deconstruction.

But deconstruction was far less tolerant than New Criticism. It said you have to read every poem as meaning one thing and its opposite. This was how the heady optimism of early New-Critical days evolved into a world-weary, endlessly recurring, formulaic self-contradiction: all texts in the end say the same self-subverting sort of thing.

Such a theory could not interest anyone very long—and indeed deconstruction was much shorter-lived than New Criticism. This explains why literature departments now have largely turned away from “readings” and have focused their work (often productively) on cultural activism and historical studies.

***

Fast-forward a decade and a half to the late 1970s. By this time, I was a chaired professor at a top-rated English department. I’d written several articles and books on English Romantic poets and theory of interpretation, and I was putting the maximum into my retirement fund. But I was getting worried: After serving two stints as department chairman, I’d seen that English programs were neglecting the task of teaching composition.

With the help of a grant from the National Endowment for the Humanities, I decided to do some large-scale empirical work on how to teach writing more effectively. Studies by the Educational Testing Service had shown that the teaching of composition was currently neither an art nor a science, but almost completely arbitrary. When a single paper was graded by multiple people, the resulting grade was unpredictable almost to the point of randomness. My research was designed to discover whether we could devise a non-arbitrary grading system based on the actual communicative effectiveness of writing.

But what I discovered was something altogether unexpected and, as it turned out, life-changing. I found that when readers were somewhat unfamiliar with the topic in the text, no paper, no matter how well written, could communicate effectively with those readers. I had assumed that clear writing would help the most when the subject was unfamiliar. In fact, the opposite was true. When the topic was familiar to readers, you could measure the benefits of good writing (and the problems caused by bad writing) quite consistently. But the time and effort it takes to understand a text on an unfamiliar topic completely overwhelms the effects of writing quality.

When we carried our experiments to a community college in Richmond, this truth became more apparent—and extremely urgent. These students, primarily from disadvantaged backgrounds, could easily read a text on “Why I like my roommate.” But even after controlling for vocabulary level and syntax, they could not easily read about Lee’s surrender to Grant. These Richmond students, surrounded by Civil War mementos on Monument Avenue, were clueless about the Civil War. Their lack of knowledge was the reason they were unable to read well about anything beyond the most banal topics.

At the same time as I was doing this research, other studies were beginning to show that relevant prior knowledge—information already stored in one’s long-term memory—is the single most important factor in reading comprehension. It’s more important than average vocabulary level, syntactic complexity, and all the other technical characteristics of texts used by schools to determine grade-appropriate texts.

Schools continue to give the impression that there is such a thing as a general level of reading skill. One student is said to be reading on grade level, while another is said to be some precise number of grade levels ahead or behind. All of this makes sense when talking about decoding skills—the ability to translate those marks on the page into words. But when it comes to reading comprehension, there is no such thing as a general level of reading skill. That single score that a student receives on a test masks the fact that the test itself had a variety of passages on a variety of topics. When the content in a passage is familiar, students read it well. When it is unfamiliar, they read it poorly.

Decades of cognitive science research boil down to this: For understanding a text, strategies help a little, and knowledge helps a lot. I consider this the single most important scientific insight for improving American schooling that has been put forward in the past half century. But unless one is familiar with the research, it’s hard to overcome the cast of mind that regards reading and writing as a set of technical skills—just as devotees of the New Criticism had done.

***

When I first started my experiment on writing, I thought it would prove that a student could become a good writer by learning a few formal techniques. But the data showed that background knowledge, not technique, is by far the more important element in both writing and reading. Technique only gets you about 10 per cent of the way in communication. The remaining 90 percent requires knowledge—knowledge that those struggling readers in Richmond hadn’t been taught.

When the results of our writing experiment surprised us, an unprepared mind might have simply considered it a failed experiment. I realized years later that it was my own prior knowledge that allowed me to comprehend the results of the study. The light bulb went on for me only because my mind had been prepared by my work in literary theory: the harsh glare of a bright-yet-contentedly undereducated student and the contradictory interpretations of two poems.

Fundamentally, the Core Knowledge reform movement is an effort to give all students the broad knowledge that will set them up for a good income and a lifetime of reading and learning. I won’t be around to see how it ends. With luck it could end with higher achievement and much smaller achievement gaps—but only if far more schools, parents, and concerned citizens become persuaded, as I did, that knowledge trumps skills.

Happy 85th Birthday E. D. Hirsch, Part 1: The Secret to Lifelong Learning

by Lisa Hansel
March 19th, 2013

Tags: ,
Posted in Curriculum, Education Practice, Education Theory, Research and Reports | 40 Comments »

On Friday, March 22, E. D. Hirsch turns 85. He’s celebrating with a typical week of researching ideas for increasing educational equity, being interviewed, writing, and spending time with family. I’m celebrating by rereading many of his books and essays—and sharing the highlights in a series of posts. Please join the festivities by adding warm birthday wishes, and your favorite Hirsch quotes, in the comments.

To kick off this fête, here’s a selection from The Schools We Need and Why We Don’t Have Them, which Hirsch thinks is his most effective attempt to make the case for a knowledge-building curriculum. Drawn from pages 152 – 158,* this addresses the question “What Is Higher-Order Thinking?”

Two traditions in cognitive psychology are useful for understanding the nature of the critical-thinking, problem-solving skills that we wish to develop in our students. One tradition has studied the characteristic differences between expert and novice thinking, sometimes with the practical goal of making novices think more like experts as fast possible. Another tradition has investigated the differences between accurate and inaccurate thinking of the everyday newspaper-reading, bargain-hunting sort that all of us must engage in as nonexperts. Both sorts of study converge on the conclusion that, once basic underlying skills have been automated, the almost universal feature of reliable higher-order thinking about any subject of problem is the possession of a broad, well-integrated base of background knowledge relevant to the subject. This sounds suspiciously like plain common sense (i.e., accurate everyday thinking), but the findings entail certain illuminating complexities and details that are worth contemplating. Moreover, since the findings run counter to prevailing fact-disparaging slogans of education reform, it will be strategically useful to sketch briefly what research has disclosed about the knowledge-based character of higher-order thinking.

The argument used by educators to disparage “merely” factual knowledge and to elevate abstract, formal principles of thought consists in the claim that knowledge is changing so rapidly that specific information is outmoded almost as soon as it has been learned. This claim goes back at least as far as Kilpatrick’s Foundations of Method (1925). It gains its apparent plausibility from the observation that science and technology have advanced at a great rate in this century, making scientific and technological obsolescence a common feature of modern life. The argument assumes that there is an analogy between technological and intellectual obsolescence. Educators in this tradition shore up that analogy with the further claim that factual knowledge has become a futility because of the ever-growing quantity of new facts. The great cascade of information now flowing over the information highway makes it pointless to accumulate odd bits of data. How, after all, do you know which bits are going to endure? It is much more efficient for all students to spend time acquiring techniques for organizing, analyzing, and accessing this perpetual Niagara of information.

Like the tool metaphor for education, the model of acquiring processing techniques that would be permanently useful—as contrasted with acquiring mere facts that are soon obsolete—would be highly attractive if it happened to be workable and true. But the picture of higher thinking skills as consisting of all-purpose processing and accessing techniques is not just a partly inadequate metaphor—it is a totally misleading model of the way higher-order thinking actually works. Higher thought does not apply formal techniques to looked-up data; rather, it deploys diverse relevant cues, estimates, and analyses from preexisting knowledge. The method of applying formal techniques to looked-up data is precisely the inept and unreliable problem-solving device used by novices. As a model of real-world higher-order thinking, the picture is not simply inaccurate—it reverses the realities. It describes the lower-order thinking of novices, not the higher-order thinking of experts.

A useful illustration of the point is presented by Jill Larkin and Ruth Chabay in a study of the ways in which novices and experts go about solving a simple physics problem. The problem Larkin and Chabay set up is (in simple terms) to find out how much friction there is between a sled and the snow-covered ground when a girl is pulling her little brother through the snow at a constant rate. The brother and the sled together weight 50 pounds. The sister is pulling with a force of 10 pounds, and she pulls the rope at an angle of 30 degrees from the horizontal. What is the coefficient of friction? The typical novice tries to solve the problem by applying formal equations that can be looked up…. The [resulting] answer is wrong, not because the equation or math is wrong but because the novice does not know enough about real-world physics to know how to connect the formula to the problem….

The expert physicist goes about the problem differently. He or she analyzes the critical components of the situation before looking up equations, and makes two critical observations before even bothering with numbers. The first observation is that the sled is going at a constant speed, so that, in effect, there is no net residue of forces acting on the sled; there is an exact balance between the force exerted horizontally by the girl’s pull and he force exerted against that pull by friction. If there had been some difference in the two forces, then the sled would speed up or slow down. So the answer has got to be that the friction is exactly equal to the horizontal component of the force exerted by the girl. The physicist also sees that since the rope is pulled at 30 degrees, part of the girl’s 10 pounds of force is vertical. The answer is going to be that the friction equals the horizontal force of the girl’s pull, which is going to be that 10 pounds minus its vertical component. The structure of the answer is solved on the basis of multiple cues and relevant knowledge, before any formulas are looked up and applied. Larkin and Chabay make the following comment (which is much more to our purpose than the details of the physics involved):

Scientists’ problem solving starts with redescribing the problem in terms of the powerful concepts of their discipline. Because the concepts are richly connected with each other, the redescribed problem allows cross checking among inferences to avoid errors. [My emphasis.]

An important feature of higher-order thinking is this “cross checking among inferences,” based on a number of “richly connected” concepts. In higher-order thinking, we situate a problem in mental space on analogy with the way we situate ourselves in a physical space—through a process of cross-checking or triangulation among relevant guideposts in our landscape of preexisting knowledge. If we look at a problem from a couple of different angles, using a couple of different cues, and if our different estimates converge, we can gain confidence in our analysis and can proceed with confidence. If, on the other hand, there is some dissonance or conflict between our cues, then warning signals go up and we figure out which approach is more probable or fruitful. The procedure is clearly a very different and far more reliable mode of thinking than the error-prone method of applying formal techniques to looked-up data.

The example also illustrates the implausibility of the claim that school-based information quickly grows outdated. How outmoded will the knowledge used to solve the sled problem become? A philosopher of science, Nicholas Rescher, once observed that the latest science is in a sense the least reliable science, because, being on the frontier, it is always in dispute with other, rival theories—any of which may emerge victorious. Accordingly, reasoned Rescher, the most reliable physics is “stone-age physics”: if you throw the rock up, it is going to come down. For most problems that require critical thought by the ordinary person regarding ethics, politics, history, and even technology, the most needed knowledge is usually rather basic, long-lived, and slow to change. True, just as physics is under revision at the frontier, so American history before the Civil War is constantly under revision in certain details (e.g., did Abraham Lincoln have an affair with Ann Rutledge?). But behind the ever-changing front lines, there is a body of reliable knowledge which has not changed, and will not change very much, and which serves very well as a landscape to orient us in mental space. It is true that, over time, the content of the most significant and useful background knowledge for today’s world does change. But I have never seen a carefully reasoned defense of the repeated assertion that, in the new age, factual knowledge is changing so fast as to make the learning of significant information useless. Probably, no carefully reasoned defense of this mindless claim could be mounted….

The key trait to remember about higher-order thinking is its mixed character, consisting of operational facility and domain-specific knowledge….

The best research on this subject shows that neither fact-filled memorization nor large conceptual generalizations are effective modes of education for higher-order thinking about the complexities of the modern world. On the other hand, it has been shown that accurate factual estimates are necessary for understanding many issues. Norman Brown and Robert Siegler summarize the underlying problem for modern education:

Faced with the issue of how to inculcate such information, educators have oscillated between two approaches. One has been to require students to memorize large numbers of quantitative facts. The other has been to deemphasize dates, magnitudes, and other quantities, and to focus on understanding qualitative relations. Each of these approaches has major drawbacks, however…. There are just too many such facts for anyone to memorize a high percentage of them. On the other hand, it is difficult if not impossible to acquire more than a superficial understanding of a domain without some degree of quantitative sophistication about it.

The breadth-depth issue will always be with us, and will always require compromises and common sense….

Research has demonstrated that the teaching of a generous number of carefully chosen exemplary facts within a meaningful explanatory context is a better method for inducing insightful thinking than is any proposed alternative….

This finding has strong implications for curriculum making. The conclusion from cognitive research shows that there is an unavoidable interdependence between relational and factual knowledge, and that teaching a broad range of knowledge is essential to effective thinking both within domains and among domains….

A wide range of knowledge and a broad vocabulary supply entry wedges into unfamiliar domains, thus truly enabling “lifelong learning,” as well as the attainment of new knowledge and greater depth as needed. The unmistakable implication for modern education is that, instead of constantly deferring the introduction of challenging and extensive knowledge, we need to be taking the opposite tack by increasing both the challenge and the breadth of early education.

 

* For the endnotes, please refer to the book.

 

You may also be interested in the other posts in this birthday retrospective:

Part 2: Avoidable Injustice

Part 3: Breaking Free from the Siren Song

Part 4: Passing the Test

 

 

Mere Facts, Mere Knowledge, Mere College Readiness

by E. D. Hirsch, Jr.
February 25th, 2013

Tags: ,
Posted in Education Theory, Research and Reports | 13 Comments »

Is teaching many domains in English language arts more important to college and career readiness than teaching many words?

Research on teaching vocabulary has determined better and worse ways of conducting explicit instruction.  Word lists and isolated definitions, while they may seem efficient, are among the least effective methods, while explicit explanations of words in context are the most effective. Ideally, according to one distinguished researcher, students can learn up to 400 new words in a school year by explicit methods (2+ words a day for 180 days under ideal circumstances). Others offer a more modest estimate, around 200 words per school year.

Yet the minimal count of words you need to be college and career ready is estimated to be 12,000 to 30,000, depending on the mode of counting. The explicit method of instruction at its best yields 5,200 words between kindergarten and 12th grade. Yet even marginal high school students need to know twice that many—meaning that most of their word learning must occur incidentally in the course of understanding the gist of spoken and written language.

Nonetheless, I would agree with advocates of explicit word study that, done strategically as an integrated and not-very-time-consuming part of a lesson, explicit instruction can help unlock enough of the gist of a passage to speed up the incidental learning of words. But then the question arises: what sort of words should we pause over in order to make the best use of class time and help the student make the fastest progress?

Experts in explicit word study have identified three main categories of words called Tier 1, Tier 2, and Tier 3, in order of frequency of occurrence in written English. The current expert view is that teachers should focus on Tier 2 words. Tier 1 words are so usual that students are likely to learn them on their own. Tier 3 words, on the other hand, are so rare that focusing on them does not offer much advancement for general reading ability. So under current thinking, the following sorts of Tier 2 words are the ones teachers should spend most class time on:  reputation, disruption, hovers, stifling, obstacle, descendants, maximum, standards, barren, desolate—words that are moderately frequent, because used in multiple written contexts. That’s not true of domain-specific Tier 3 words like, valence, bildungsroman, Renaissance, metabolism, Gettysburg, photosynthesis,  stochastic, ionic, simile, dew point, polygon, Madison, monotheism, kinetic, Dalton, Fourier, Magna Carta, Impressionism, helium, fiscal, TR, and Shiite. 

But I’m not persuaded by this rationale. Although Tier 2 words are to be found in multiple contexts, they do not constitute a big percentage of the totality of different words in the English vocabulary. That distinction belongs to the words of Tier 3, which are domain specific. If you want to reach the magic number of 25,000 thousand or so, it’s best to spend your time learning domain-specific Tier 3 words. After all, there’s a bit of inconsistency in the expert advice to teachers to spend most of students’ explicit-word-study time on Tier 2 words after having said that Tier 1 words can be ignored on the grounds that they are used so frequently that most people have learned them incidentally. That sensible principle recedes when it comes to their doctrine about Tier 2 words, which we are advised to focus on precisely because they are relatively frequent and are used in multiple written contexts. Some serious research needs to be undertaken to determine whether, in a good, coherent knowledge-based curriculum most Tier 2 words aren’t also learned incidentally as a matter of course, just like most Tier 1 words, as the overall math would suggest. (This research has not been conducted, despite the confident advice about studying domain-general Tier 2 words. Indeed there is some counter evidence in the studies by John Guthrie indicating the superiority of domain-specific instruction in ELA.)

To support the emphasis on Tier 2 words many educators assume that there exists such a thing as general “reading skill,” which will be the key to college and career readiness. But cognitive scientists instruct us that it’s an oversimplification to suppose that there is such a thing as a domain-general reading skill that can be fostered by the explicit study of domain-general, Tier 2 words. On the contrary, the latest cognitive science tells us that reading skills, like most skills, are “domain specific.”  Granted, there are important domain-general aspects of reading that include automatic, unconscious procedures like decoding skill, eye movements, strategic meaning searches, and knowledge of domain-general words. It is reasonable, indeed essential, to ensure that students gain such domain-general knowledge. But few experts advise that students be explicitly trained in eye-movement patterns, at least not very extensively. For most students that skill develops unconsciously without continuous instruction.  The same is true of most domain-general word learning—which occurs unconsciously, bit by bit, through multiple exposures to a word in different contexts. Domain general skills like decoding, once mastered, are continually practiced and unconsciously improved precisely because, being domain general, they occur frequently.

There’s a clear analogy with skill in sports. Most sports demand domain-general athletic abilities like hand-eye coordination.  Nonetheless being skilled specifically in golf does not directly transfer to being skilled in tennis or even in croquet. Each sport has domain-specific skills that must be explicitly mastered. Similarly, being skilled in reading about golf does not readily transfer to being skilled in reading about tennis. The golf passages will of course contain domain-general words like but, however, pretty, and willing, but the critical words will be birdie, bogie, and par, and knowing them won’t help you read a tennis story with set point, fault, and ace.

Why do you suppose school reading tests typically offer ten or so passages? If reading were a domain-general skill, one passage would suffice. (If I want to know if you can ride a bike, I won’t bring ten bikes for you to ride. One will suffice.) But reading tests always contain several passages because a reliable reading test has to sample your ability to read in several different domains. Reading tests are essentially tests of how many different domains you have knowledge of and vocabulary for. To be a literate adult—one who could read a newspaper front to back—you must have knowledge in a very broad range of domains.

If we wish our students to perform well on a reading test, we ought to abandon the disparagement of “mere facts.”  Nothing contributes more to a student’s reading abilities than wide knowledge of multiple domains, automatically accompanied by knowledge of many domain-specific, Tier 3 words. In sum, nothing contributes more to college and career readiness than broad general knowledge over multiple domains.

The best way to teach “English language arts” then is systematically to teach substantive domains of knowledge along with their inherently related vocabularies. In fact the whole issue needs to be broadened by a return to real classes in history, science, and the arts in elementary grades, as the best way to gain proficiency in reading. This larger principle transcends the currently debated topic of fictional vs. non-fictional genres. Much good fiction is a repository of domain knowledge—not just of human nature and ethical principles, but also of historical and factual knowledge, including such things as Mississippi river-boating in Huckleberry Finn, and whaling in Moby Dick, as well as the forms and techniques of literature, like simile and metaphor, prefixes and suffixes, which are just as “informational” as chemical valences. What is needed for college and career readiness is extensive general knowledge over multiple domains, coherently delivered—with lots of Tier 3 words.

When this is done well, with gradually increasing sophistication grade by grade, Tiers 1 and 2 will mostly take care of themselves.

The Two Americas Continued: Schmidt and McKnight

by E. D. Hirsch, Jr.
February 15th, 2013

Tags: , , , ,
Posted in Education Theory, Educational Policy | 15 Comments »

In my last post (“Antonio Who?”) about the great Italian thinker Antonio Gramsci (pronounced gram-shee who was cited as a big influence by Michael Gove the British Secretary of Education), I hinted—but didn’t venture to say—that maybe our educational systems would be in better shape if our top authorities followed Gove’s lead and read more challenging books, while holding fewer committee meetings. Albert Shanker, the brilliant union-organizer-turned-educational-statesman, once told me, mournfully: “They don’t read.” I once looked through Al Shanker’s own library now housed at AFT headquarters, and was amazed to see his annotations in a multi-volume set by the philosopher Bernard Bosanquet. But the vision of a top American official reading the Prison Notebooks of Gramsci could happen only in a Woody Allen movie.

On the other side, an argument against reading a lot of books on American education is that it could cause clinical depression. As I peruse the important book by William Schmidt and Curtis McKnight, Inequality for All, I think to myself: this is a companion volume to a whole spate of recent books on American inequality by eminent scholars, including a book by the Nobel Prize winner Joseph Stiglitz, and a mournfully vivid book by Charles Murray, Coming Apart. The image one gets from Murray is not of red America vs. blue America, but of one zip code full of striving SAT-takers and community-minded citizens vs.  a neighboring zip code of drifting alcoholic semi-literates who lack any sense of community or hope. It is not too much of a stretch to see Inequality for All as identifying a significant cause of these economic and sociological ills. The book is an indictment of the content-incoherence of our schools.

The sad reality is that the American educational system does not provide equal opportunity for all but rather perpetuates vast inequalities in content coverage…. This inequality of opportunity … disadvantages many, perhaps even most, children in the United States….

Variation in content coverage corrupts the entire U.S. educational system, in effect creating an enormous educational lottery in which every student takes part—whatever their racial, ethnic, or socioeconomic background. The system of schooling represents a game of chance that few are even aware is being played.

Given this almost universal curricular incoherence in our schools, students with home advantages are able to overcome ineffectual schooling through home tutoring either direct or indirect. In short, (as Gramsci predicted) the “progressive” American theory of education, with its how rather than what approach to schooling, while it is “advocated as being democratic, is destined not merely to perpetuate social differences but to crystallize them in Chinese complexities” (Notebook 29).  In other words Gramsci predicted the very America described by Stiglitz and Murray as being the effect of the schooling described by Schmidt and McKnight.

Inequality for All focuses on math and science education, showing with authoritative thoroughness the failure of our schools to bring rationality and cumulativeness in the topics taught from year to year.   They make the point that reformers on both the left and right have been consumed with equalizing resources or in fostering competition and accountability, but pay too little heed to the essence of schooling which they see as the delivery of academic content by teachers to students. Hear, hear!

I have tried to make exactly the same point with respect to the general knowledge that students need to gain outside the subjects of science and math. The Gramsci principle that the delivery of academic content is the key to social justice holds even more strongly for general knowledge, which is the key to high literacy and the ability to learn and adapt in the future. Indeed, based on data from the Armed Forces Qualification Test, I’ve argued that general knowledge is approximately twice as important as math in determining a person’s future capacity to function economically and as a citizen, and therefore deserves at least the same care and coherence that Schmit and McKnight want for science and math.  Given their sound view that “the delivery of academic content” is the key to future improvement and to equity, Schmidt and McKnight come out strongly in favor of the Common Core State Standards.

Every day my email inbox fills with relentless attacks on these standards, and renewed attempts to undo the commitments of forty-odd states to follow them. I wish these energies and criticisms could be turned to making the standards function well, rather than to making them go away. They are a work in progress, and instituting them will entail many false steps. But Schmidt and McKnight rightly see them as the best way forward for excellence and equity. They see the issue in educational, not political terms: not as some intolerable imposition of the federal government or the Gates Foundation, but as our best chance to overcome failure, incoherence, and injustice.

Unless the carpers against the common core can come up with an alternative plan that brings coherence to “the delivery of academic content,” they leave us in the unacceptable condition of the status quo.   Let these carpers produce a book half as thorough and authoritative as that of Schmidt and McKnight, with a vision of what needs to be done half as compelling. Then I might be more receptive to their constant stream of mosquito bites against the ambitious vision defended in this important book.

 

Antonio Who?

by E. D. Hirsch, Jr.
February 13th, 2013

Tags: , ,
Posted in Education Theory, Educational Policy | 22 Comments »

Michael Gove, the British Secretary of State for Education, is a man who reads serious books on education and follows their arguments. In a remarkable speech the other night, he mentioned some of the intellectual influences that have caused him to shake up the British educational world by insisting that students begin learning facts again. One of those influences was our own Daniel Willingham, and he even quoted from a 1996 book by me. But he said that the greatest intellectual influences on his educational thought were the writings of Antonio Gramsci. So here we have a Tory cabinet minister singing the praises of one of the most revered Communist thinkers of the 20th century. What gives?

I don’t doubt that Michael Gove might have an impish sense of humor and take pleasure in suggesting to his shadow opponents in the Labour party and in the anti-fact party of educators: “Look I’m just supporting what the most profound leftist thinker of the 20th century had to say about education.” But Gove’s main aim was deadly serious. Gramsci was an astonishingly prescient and penetrating thinker whose work is all the more remarkable since it was written under depressing conditions—in prison, where he languished because his writing and journalistic work in the 1920s were so cogent and influential that Mussolini’s fascistic regime seized him in 1927 with the avowed purpose of silencing him. There he remained for eight years, until his ill health brought him to a sanitarium in 1934, and to a clinic in 1937, where he died. He was allowed to write, but not, of course, to let anyone see his writing. It’s only because his sister-in-law, visiting his clinic room in 1937, smuggled out his 33 prison notebooks, unpublished until after the war, that we know some of Gramsci’s profound ideas about society, politics, and education.

He rightly predicted that in the future, most work would entail intellectual work, and that political and economic power would reside with the educated. Especially notable was his critique of progressive education, which became the official educational doctrine of the fascist regime. Despite progressivism’s high claims to “child-centered natural development,” “deep understanding,” and “independent thought,” its anti-bookish tendencies, Gramsci said, were socially retrograde. “Il bambino non è un gomitolo di lana da sgomitolare, ma la parte del complesso mondo storico su cui l’ambiente e la società esercitano la loro coercizione”. “The child is not a ball of yarn to be unwound, but part of a complex historical world in which the environment is a society that exercises its own coercions.” Under progressivism, the children of the rich would continue to possess the knowledge they needed to wield the levers of power (because they would always have multiple opportunities for bookish learning), while the children of the poor would remain in their subordinate poverty.

Hence, what was needed, Gramsci said was a single “formative school” for all students rich or poor that would stress foundational knowledge in literature, science, history and the arts, in a demanding common curriculum. Only in later grades should there be practical trainings in technical and job related subjects. What Gramsci was in fact proposing was the American Common-School idea of the 19th century. And in fact his scuola formativaunica is sometimes translated as “common school.”

In sum, Gramsci favored the kind of knowledge-based schooling that Michael Gove is proposing. He would also favor the Common Core State Standards in the United States, so long as these were implemented as a specific knowledge-based curriculum, and were freed from the anti-intellectual and socially retrograde effects of what Gramsci disdainfully called “teoria dello sgomitolamento”—“the unravelling theory,” best translated as “constructivism”—the anti-broad knowledge, anti-guided learning theory that still dominates many teacher training schools in the United States.

An Unclaimed Lottery Ticket

by E. D. Hirsch, Jr.
January 17th, 2013

Tags: ,
Posted in Education Theory, Essay | 18 Comments »

Inspired by Joseph Stiglitz’s The Price of Inequality and Timothy Noah’s The Great Divergence, which lay out in disheartening detail the growing inequality of income and opportunity in the United States, I have a new article in City Journal: “A Wealth of Words.” I hope you’ll find time for the whole article, but here’s my CliffsNotes version.

With the decline of the middle class, the aristocracy of family so deplored by Jefferson seems upon us; the counter-aristocracy of merit that long defined America as the land of opportunity has receded. But there is a road back to the City upon a Hill.

There’s a positive correlation between a student’s vocabulary size in grade 12, the likelihood that she will graduate from college, and her future level of income. The reason is clear: vocabulary size is a convenient proxy for a whole range of educational attainments and abilities—not just skill in reading, writing, listening, and speaking but also general knowledge of science, history, and the arts. If we want to reduce economic inequality in America, a good place to start is the language-arts classroom.

The correlations between vocabulary size and life chances are as firm as any correlations in educational research. Of course, vocabulary isn’t perfectly correlated with knowledge. People with similar vocabulary sizes may vary significantly in their talent and in the depth of their understanding. Nonetheless, there’s no better index to accumulated knowledge and general competence than the size of a person’s vocabulary. Simply put: knowing more words makes you smarter. And between 1962 and the present, a big segment of the American population began knowing fewer words, getting less smart, and becoming demonstrably less able to earn a high income.

The sociologist Donald Hayes, following the lead of the great literacy scholar Jeanne Chall, found that publishers, under the influence of progressive educational theories, had begun to use simplified language and smaller vocabularies.

If vocabulary is related to achieved intelligence and to economic success, our schools need to figure out how to encourage vocabulary growth. They should understand, for starters, that word-learning occurs slowly and through a largely unconscious process. Consider the word “excrescence.” Few know the word; fewer still encounter it in their everyday lives. Maybe you do know it, but imagine that you don’t.

Now suppose I gave it to you in a sentence: “To calculate fuel efficiency, the aerospace engineers needed an accurate estimation of excrescence drag caused by the shape of the plane’s cabin.” That single exposure to the word is probably insufficient for you to grasp its meaning, though if you know something about aerospace engineering, you’ll be likelier to make a good approximation. Here’s an encounter in another context: “Excrescences on the valves of the heart have been known to cause a stroke.” Perhaps now you have a vague understanding of the word. A third meaningful encounter will allow you to check your understanding or refine your sense of the meaning: “The wart, a small excrescence on his skin, had made Jeremy self-conscious for years.” By now, you probably have a pretty solid understanding of the word, and one more encounter in a familiar context should verify your understanding: “At the far end of the meadow was what, at first glance, I thought a huge domed building, and then saw was an excrescence from the cliff itself.”

You’ve probably figured out that the word “excrescence” means “an outgrowth.” That’s an accelerated, artificial example of how word-learning occurs. Almost all the word meanings that we know are acquired indirectly by intuitively guessing new meanings as we get the overall gist of what we’re hearing or reading.

The context for an unfamiliar word isn’t just the other words surrounding it in a text but also the situation referred to by those words. Familiarity with the relevant subject matter ensures that a student’s unconscious meaning-guesses are likely to be right.

So the fastest way to gain a large vocabulary through schooling is to follow a systematic curriculum that presents new words in familiar contexts, thereby enabling the student to make correct meaning-guesses unconsciously. There are so many words to be learned by 12th grade—between 25,000 and 60,000—that a large vocabulary results not from memorizing word lists but from systematically acquiring knowledge about the social and natural worlds.

The idea is to immerse students in a domain long enough to make them familiar with the context—and thus able to learn words faster. For the purposes of teaching vocabulary, a “domain” could be defined as a sphere of knowledge in which concepts and words are repeated over the course of two or three weeks. Such repetition happens automatically in a classroom unit on, say, plants and photosynthesis.

I would make three practical recommendations to improve American students’ vocabularies, and hence their economic potential: better preschools; classroom instruction based on domain immersion; and a specific, cumulative curriculum sequence across the grades, starting in preschool. Of these, the last is the most important but also the toughest to achieve politically. But the new Common Core State Standards for language arts, now adopted by more than 40 states, may offer a ray of hope (see “The Curriculum Reformation”). One statement in the new standards reads: “By reading texts in history/social studies, science, and other disciplines, students build a foundation of knowledge in these fields that will also give them the background to be better readers in all content areas. Students can only gain this foundation when the curriculum is intentionally and coherently structured to develop rich content knowledge within and across grades.” A second encouraging passage: “The Common Core Standards do not—indeed, cannot—enumerate all or even most of the content that students should learn. The Standards must therefore be complemented by a well-developed, content-rich curriculum.”

These two statements are big steps forward from the failed how-to approaches of the recent past. My hope is that some influential district superintendent will require a specific grade-by-grade knowledge sequence. The striking success of one major urban district could transform practice throughout the nation.

Tucker Takes Zhao in Just One Round

by E. D. Hirsch, Jr.
January 16th, 2013

Tags: ,
Posted in Education Theory, Educational Policy | 4 Comments »

High praise for Marc Tucker’s thoughtful response to Yong Zhao’s well-meaning, but terribly misguided, critique of the Common Core State Standards (CCSS). Since I’ve decided to judge this boxing match, I’m calling a TKO. Tucker won round one so easily that it would be dangerous for Zhao to try to fight on.

Zhao reveals that he does not know much about the CCSS (he expects a new world “where all American children are exposed to the same content, delivered by highly standardized teachers”) and has been duped by left-brain/right-brain silliness (he writes that “Right-brained directed skills, including design, story, symphony, empathy, play, and meaning, will become more valuable…. [But] the core subjects prescribed by the Common Core … are mostly left-brained cognitive skills”). Tucker refutes Zhao’s key points effortlessly. I hope you’ll read their exchange, so I’ll just give two highlights:

1. Tucker on Zhao’s concern that the CCSS will quash creativity:

“The literature is clear.  Truly creative people know a lot and they have worked hard at learning it.  They typically know a lot about unrelated things and their creativity comes from putting those unrelated things together in unusual ways.  Learning almost anything really well depends on mastering the conceptual structure of the underlying disciplines, because, without that scaffolding, we are not able to put new information and skills to work.”

2. Tucker on Zhao’s concern that the CCSS will not prepare students for tomorrow’s unknown jobs:

“It is true that the future will be full of jobs that do not exist now and challenges we cannot even imagine yet, never mind anticipate accurately.  But, whatever those challenges turn out to be, I can guarantee you that they will not be met by people without strong quantitative skills, people who cannot construct a sound argument, people who know little of history or geography or economics, people who cannot write well.”

The fact is, the new standards are a big step toward preparing all students for more rigorous higher education—be it online, on a traditional college campus, or on the job. What folks like Zhao are missing is a solid understanding of the research on language comprehension, effective communication, critical thinking, and other crucial abilities (like being responsible citizens). These abilities depend on knowledge. Not just any knowledge—relevant, subject- and task-specific knowledge. The more extensive the knowledge, the deeper the analysis. The CCSS—because they bring higher-quality fiction and dramatically more nonfiction into the classroom, and because they provide a coherent, focused, and rigorous approach to mathematics—will help millions of students develop such knowledge.

Blame the Tests

by E. D. Hirsch, Jr.
January 15th, 2013

Tags: , , , ,
Posted in Assessment and Testing, Education Practice, Education Theory, Educational Policy, No category, Research and Reports | 6 Comments »

In Praise of Samuel Messick 1931–1998, Part III

The chief practical impact of NCLB has been its principle of accountability. Adequate yearly progress, the law stated, must be determined by test scores in reading and math—not just for the school as a whole, but for key groups of students.

Now, a decade later, the result of the law, as many have complained, has been a narrowing of the school curriculum. In far too many schools,  the arts and humanities, and even science and civics, have been neglected—sacrificed on the altar of tests  without any substantial progress nationwide on the tests themselves. It is hard to decide whether to call NCLB a disaster or a catastrophe.

But I disagree with those who blame this failure on the accountability principle of NCLB. The law did not specify what tests in reading and math the schools were to use. If the states had responded with valid tests—defined by Messick as tests that are both accurate and have a productive effect on practice—the past decade would have seen much more progress.

Since NCLB, NAEP’s long-term trend assessment shows substantial increases in reading among the lowest-performing 9-year-olds—but nothing comparable in later grades. It also shows moderate increases in math among 9- and 13-year-olds.

So, it seems that a chief educational defect of the NCLB era lay in the later-grades reading tests; they simply do not have the same educational validity of the tests in early grades reading and in early- and middle-grades math.

 ****

It’s not very hard to make a verbal test that predicts how well a person will be able to read. One accurate method used by the military is the two-part verbal section of the multiple-choice Armed Forces Qualification Test (AFQT), which is known for its success in accurately predicting real-world competence. One section of the AFQT Verbal consists of 15 items based on short paragraphs on different subjects and in different styles to be completed in 13 minutes.  The other section of the AFQT Verbal is a vocabulary test with 35 items to be completed in 11 minutes. This 24-minute test predicts as well as any verbal test the range of your verbal abilities, your probable job competence and your future income level. It is a short, cheap and technically valid test. Some version of it could even serve as a school-leaving test.

Educators would certainly protest if that were done—if only because such a test would give very little guidance for classroom practice or curriculum. And this is the nub of the defects in the reading tests used during the era of NCLB: They did not adequately support curriculum and classroom practice. The tests in early-grades reading and in early- and middle-grades math did a better job of inducing productive classroom practice, and their results show it.

Early-grades reading tests, as Joseph Torgesen and his colleagues showed, probe chiefly phonics and fluency, not comprehension. Schools are now aware that students will be tested on phonics and fluency in early grades. In fact, these crucial early reading skills are among the few topics for which recent (pre-Common Core) state standards had begun to be highly specific. These more successful early reading tests were thus different from later ones in a critical respect:  They actually tested what students were supposed to be taught.

Hence in early reading, to its credit, NCLB induced a much greater correlation than before between standards, curriculum, teaching and tests. The tests became more valid in practice because they induced teachers to teach to a test based on a highly specific subject matter—phonics and fluency. Educators and policymakers recognized that teaching swift decoding was essential in the early grades, tests assessed swift decoding, and—mirabile dictu—there was an uptick in scores on those tests.

Since the improvements were impressive, let’s take a look at what has happened in over the past decade among the lowest performing 9-year-olds on NAEP’s long-term trend assessment in reading.

Note that there is little to no growth among higher-performing 9-year-olds, presumably because they had already mastered phonics and fluency.

Similarly, early- and middle-grades math tests probed substantive grade-by-grade math knowledge, as the state standards had become ever more specific in math. You can see where I’m going: Early reading and math improved because teachers typically teach to the tests (especially under NCLB-type accountability pressures), and the subject matter of these tests began to be more and more defined and predictable, causing a collaboration and reinforcement between tests and classroom practice.

In later-grades reading tests, where we have failed to improve, the tests have not been based on any clear, specific subject matter, so it has been impossible to teach to the tests in a productive way. (The lack of alignment between math course taking and the NAEP math assessment for 17-year-olds is similarly problematic.) Of course, there are many reasons why achievement might not rise. But specific subject matter, both taught and tested, is a necessary—if not sufficient—condition for test scores to rise.

In the absence of any specific subject matter for language arts, teachers, textbook makers, and test makers have conceived of reading comprehension as a strategy rather than as a side effect of broad knowledge. This inadequate strategy approach to language arts is reflected in the tests themselves. I have read many of them.  An inevitable question is something like this: “The main idea of this passage is….” And the theory behind such a question is that what is being tested is the ability of the student to strategize the meaning by “questioning the author” and performing other puzzle-solving techniques to get the right answer. But, as readers of this blog know, that is not what is being tested. The subject matter of the passage is.

This mistaken strategy-focused structure has made these tests not only valueless educationally, but worse—positively harmful. Such tests send out the misleading message that reading comprehension is chiefly strategizing. That idea has dominated language arts instruction in the past decade, which means that a great deal of time has been misspent on fruitless test-taking activities. Tragically, that time could have been spent on science, humanities and the arts—subjects that would have actually increased reading abilities (and been far more interesting).

The only way that later-grades reading tests can be made educationally valid is by adopting the more successful structure followed in early reading and math. An educationally valid test must be based on the specific substance that is taught at the grade level being tested (possibly with some sampling of specifics from previous and later grades for remediation and acceleration purposes). Testing what has been taught is the only way to foster collaboration and reinforcement between tests and classroom practice. An educationally valid reading test requires a specific curriculum—a subject of further conversations, no doubt.

The Work of a Great Test Scientist Helps Explain the Failure of No Child Left Behind

by E. D. Hirsch, Jr.
January 10th, 2013

Tags: , , ,
Posted in Assessment and Testing, Education Practice, Education Theory, Educational Policy, Research and Reports | 8 Comments »

In Praise of Samuel Messick 1931–1998, Part II

In a prior post I described Messick’s unified theory of test validity, which judged a test not to be valid if its practical effects were null or deleterious. His epoch-making insight was that the validity of a test must be judged both internally for accuracy and externally for ethical and social effects. That combined judgment, he argued, is the only proper and adequate way of grading a test.

In the era of the No Child Left Behind law (2001), the looming specter of tests has been the chief determiner of classroom practice. This led me to the following chain of inferences: Since 2001, tests have been the chief determiners of educational practices. But these tests have failed to induce practices that have worked. Hence, according to the Messick principle, the tests that we have been using must not be valid. Might it be that a new, more Messick-infused approach to testing would yield far better results?

First, some details about the failure of NCLB. Despite its name and admirable impulses it has continued to leave many children behind:

 

NCLB has also failed to raise verbal scores. The average verbal level of school leavers stood at 288 when the law went into effect, dropped to 283 in 2004, and stood at 286 in 2008.

Yet this graph shows an interesting exception to this pattern of failure, and it will prove to be highly informative under Messick’s principle. Among 4th graders (age 9) the test-regimen of NCLB did have a positive impact.

Moreover, NCLB also had positive effects in math:

This contrast between the NCLB effects in math and reading is even more striking if we look at the SAT, where the test takers are trying their best:

So let’s recap the argument. Under NCLB, testing in both math and reading has guided school practices. Those practices were more successful in math and in early reading than in later reading. According to the Messick principle, therefore, reading tests after grade 4 had deleterious effects and cannot have been valid tests. How can we make these reading tests more valid?

A good answer to that question will help determine the future progress of American education. Tune in.

« Older Entries