Testing: From the Mouths of Babes

by Lisa Hansel
May 8th, 2014

“No one learns from state tests. It’s testing what you know. You’re not learning anything from it.”

 —12th grader

“I like math or spelling tests better [than state accountability tests] because you can study for them. For the [state accountability tests], I wonder what will be on them this time.”

 —5th grader

“I like pre- and post-tests because you get to see the progress you’ve made.”

—4th grader

Is it just me, or do these kids know a whole lot more about assessment and increasing educational achievement than most state and national policymakers? Far too many policymakers seem to have lost sight of the most important goal of assessment and accountability: increasing learning. They seem stuck on accountability for the sake of accountability, unwilling to ask whether assessment dollars could be used more effectively.

I’m not against accountability—and I think assessment is necessary—but I am for allocating time and money in the most effective ways. So I find these students’ thoughts, and the new study in which they appear, pretty compelling. The study is Make Assessment Matter, by the Northwest Evaluation Association in cooperation with Grunwald Associates LLC. It explores students’ (4th – 12th graders), teachers’, and administrators’ views on all sorts of testing—from classroom quizzes to state accountability tests. Conclusion: “There is an urgency felt on the part of students, teachers and district administrators to emphasize assessment for learning rather than for accountability. The overwhelming preference for all parties is that assessment results be used to inform learning.” Sadly, today’s state tests not only don’t inform learning, they seem to be impeding it: “teachers (70 percent) and district administrators (55 percent) … [say] that the focus on state accountability tests takes too much time away from learning.”

Think about the weeks that are lost to state accountability tests each year as you absorb these key findings:

On the one hand, the vast majority of students, boys and girls, say they try hard on most tests and care about doing well on tests, among other findings that indicate how seriously they take tests and learning. On the other hand, some boys (46 percent) and girls (39 percent) say that tests are a waste of time.

It’s clear that students feel that certain kinds of tests are not very relevant to their learning, and so it’s not surprising to hear some students identify tests as a waste of time. In tandem with other findings, the message is clear: students want high-quality, engaging assessments that are tightly connected to learning….

Like students, teachers and district administrators would prefer to focus on tests that inform student learning. Most teachers (54 percent), and the vast majority of district administrators (89 percent), say that the ideal focus of assessments should be frequently tracking student performance and providing daily or weekly feedback in the classroom. This sentiment tracks with students’ attitudes about tests. Students express overwhelming agreement that tests are important for helping them and their teachers know if they are making progress in their learning and for understanding what they are learning.

Teachers say that teacher-developed classroom tests, performance tasks and formative assessment practice work best for supporting student learning in their classrooms, while state accountability tests are the least effective.

For an assessment to matter, it has to be directly tied to what is being studied in the classroom. For students to care about it, they need to be able to study for it and use the results in meaningful ways.

shutterstock_190861118

Image courtesy of Shutterstock.

That sounds perfectly reasonable to me. So, what are the logical implications for states? I see two options. One is to use Advanced Placement as a model: create detailed, content-specific courses and develop tests that only assess material in the course. I know it’s unheard of in state accountability testing, but I am actually being so crazy as to say that states should test students on the topics, books, people, ideas, events, etc. that they have been taught.

If state policymakers can’t stomach the idea of specifying the content to teach and test—if they can’t honor students’ desire to be tested in ways that inform learning—then they must honor students’ desire to not have their time wasted: make the tests zero stakes with zero test prep (like NAEP). Any test that is not tied to the specific content being studied in the classroom is a test of general knowledge and skills. Such a test can provide an informative snapshot of students’ and schools’ relative performance (and thus which schools and communities are in need of added supports). It can’t, however, indicate how any one student acquired her knowledge and skills (could be the teacher, the tutor that mom hired in October, the soccer coach who demands higher grades, the new librarian in town, finally being given eyeglasses, etc.). And therefore it can’t offer any precise indication of either teacher quality or how the student could improve. If a state wants to give a test that measures general abilities and provides nothing more than a snapshot and a trend line, that’s fine—provided the stakes and the prep time are minimized.

My preference, obviously, is for option one—especially if states would have the good sense to involve hundreds of educators in developing the specific content to be taught and assessed. Not only would the state-controlled, culminating test be useful for learning, in preparing for it teachers could use effective practices like frequent quizzing on essential content.

 

With Knowledge, You’re Never Lost

by Lisa Hansel
April 30th, 2014

Like many in the education world, I spend much of April and May wondering about U.S. testing and accountability policies. When you add up the time spent on benchmark testing, test-prep (especially the part devoted to item format, as opposed to reviewing essential knowledge and skills), testing, and grading, is that really a good use of our school days? Data to inform instruction and decision making are necessary, but do teachers, administrators, or even policymakers get what they need from the tests currently being given (or developed)? Is there a better way to improve educational outcomes?

Pretty much every article on testing this season has sent my mind spinning down these paths. I mean, really, who thought NCLB would still exist in 2014? We knew from the get-go that 100% proficient was a fairy tale. Can we please start thinking about education policy realistically? Can we be brave enough to offer clarity as to what we are trying to accomplish? With that clarity, can we find the fortitude to proceed patiently, rationally, and supportively so that (like more reasonable nations) we can attain our goals?

Probably not. But I’m not ready to give up.

Core Knowledge has a specific answer as to what it is trying to accomplish. So do all schools that have taken the time to create—and that make ongoing investments in enhancing—a detailed curriculum. Such schools should also be able to devise tests that support teaching and learning, as well as produce evidence of their educational accomplishments.

Logically, this is where I should explain the evidence that a Core Knowledge-based curriculum is effective. You can follow the link if you want. I’d rather offer an example. Call it an anecdote. Dismiss it. In a different frame of mind I might to the same. But in testing season, this example resonates with me. It’s a reminder that knowledge is inherently useful and valuable.

So, how do I know Core Knowledge works? Because it worked for an engaging lady from South Carolina who took the time to handwrite a three-page letter to E. D. Hirsch:

I am ninety-four years old and have wondered all my life about the answers to a few questions…. One being, if I am alone and lost in the woods and there is no sun and I have no compass, how do I know where North is?

shutterstock_62263246

Image courtesy of Shutterstock.

When she asked her librarian, she received a clear answer in What Your Second Grader Needs to Know, which is in the preschool through sixth-grade series Hirsch wrote for parents.

I was so impressed that I read the whole book, then asked the librarian for Book One. When finished I asked for Book Three, then Four, then Five and finished with Book Six. I was not so fortunate as the children of today, having been brought up during the Depression when I had to quit the Sophomore year of High School and go to work….

I have learned so much more from your books that my grade school didn’t teach me. I wish that all elementary schools were required to use Core Knowledge Series. Every subject was so enlightening, interesting, and helpful for me. Thank you.

How would this ninety-four-year-old student do on a standardized test, even one for fifth or sixth graders? I don’t know (she’d probably need several days of drilling on educationally irrelevant test-taking strategies). But I know she’ll never be lost in the woods. And—more importantly—I know that because she read the first through sixth grade books, she has the broad knowledge she needs to learn more each day. To grasp the scientific, political, economic, and arts topics covered in her chosen news source. To have lively debates with her neighbors. To be a well-informed voter. To show her great-grandchildren that knowledge is enlightening, interesting, and helpful.

Will The SAT Overhaul Help Achieve Equity?

by Guest Blogger
April 24th, 2014

By Burnie Bond

Burnie Bond is the director of programs at the Albert Shanker Institute. This post originally appeared on the Shanker Blog on April 22, 2014.

The College Board, the organization behind the SAT, acknowledges that historically its tests have been biased in favor of the children of wealthy, well-educated elites—those who live in the best zip codes, are surrounded by books, go to the best-regarded schools (both public and private), enjoy summer enrichment programs, and can avail themselves of as much tutoring and SAT test-prep coaching as they need. That’s why, early last month, College Board president David Coleman announced that the SAT would undergo significant changes, with the aim of making it more fair and equitable for disadvantaged students.

Among the key changes, which are expected to take effect in 2016, are: the democratization of access to test-prep courses (by trying to make them less necessary and entering into an agreement with the Khan Academy to offer free, online practice problems*); ensuring that every exam includes a reading passage from one of the nation’s “founding documents,” such as the Declaration of Independence or the Bill of Rights, or from one of the important discussions of such texts, such as the Rev. Dr. Martin Luther King Jr.’s “Letter From Birmingham Jail”; and replacing “arcane ‘SAT words’ (‘depreciatory,’ ‘membranous’),” with words that are more “commonly used in college courses, such as ‘synthesis’ and ‘empirical.’” (See here.)

Will this help? Well, maybe, but the SAT’s long heldbut always elusive—mission to help identify and reward merit, rather than just privilege, will only be met insofar as its creators can be sure that all students have had an equal opportunity to learn these particular vocabulary words and have read these particular “founding documents” and texts. That is, it comes down to a question of curriculum.

Curriculum and Equity

The connection between curriculum and equity first occurred to me when I was eight years old (though obviously not in those exact terms). For some reason, my school decided that all third graders needed to have an IQ test. I was sick that day, so one school holiday I found myself filling in bubbles alone in a classroom with Mrs. Beagles, the school’s assistant principal.

All was well until I got to one particular question. Since the test designers couldn’t be sure we could read well, many of the questions were in picture form. This one included a series of line drawings. As I recall, the first was a drawing of a boy in a ski jacket standing on a beach; the second showed a boy in swim trunks and a beach ball standing in the snow near a snowman; the third had the same swim-trunked, beach ball kid standing in sand near a big cactus; and the fourth had the ski jacket boy standing near the snowman. The question was: Which one didn’t belong?

Although I knew the “right” answer, I found myself wondering how they could just assume that I should. Having never left the tropical island of St. Croix, I had not yet been in winter or seen snow or a snowman. And, although some cactus varieties could be found out on the island’s East End, we had no real desert either. Our textbooks had not yet covered the relevant units on physical geography, and my book-loving father had only allowed a television into our house about six months beforehand. I then started wondering how many of my classmates might have thought that the ski jacket was some elaborate water flotation outfit, and how many would have been confused because we all regularly swam at East End beaches with cacti in plain sight. For that matter, what about kids on the mainland who grew up in cities or in the Midwest and who had never been to a beach or seen a desert?

Irate over the unfairness of it all, I complained to Mrs. Beagles, who replied, “Just do the best that you can,” and returned to grading papers.

I found myself thinking about this episode as I read a very interesting 2012 paper by Santelices and Wilson, whose research gave credence to an earlier paper by Freedle (also here)—the upshot of which is that the SAT Verbal continues to be biased against poor and minority students in a very particular way. That is, test takers who are African American, Hispanic-American, Asian American, or White from low-income households tend to do disproportionately well on the “hard” questions and disproportionately poorly on the “easy” ones.

In his 2003 Harvard Educational Review article, Freedle explains:

A culturally based interpretation helps explain why African American examinees (and other minorities) often do better on many hard verbal items but do worse than matched-ability Whites on many easy items. To begin with, easy analogy items tend to contain high-frequency vocabulary words while hard analogy items tend to contain low-frequency vocabulary words (Freedle & Kostin, 1997). For example, words such as “horse,” “snake,” “canoe,” and “golf” have appeared in several easy analogy items. These are words used frequently in everyday conversations. By contrast, words such as “vehemence,” “anathema,” “sycophant,” and “intractable” are words that have appeared in hard analogy items, and do not appear in everyday conversation (Berger, 1977). However, they are likely to occur in school-related contexts or in textbooks.

In other words, kids who are somewhat outside of the cultural mainstream do less well on items built around assumptions about common knowledge—the words and ideas that are “used frequently in everyday conversations.”  But what if your language or culture or social standing diminishes the chances that you actually engage in everyday conversations about golfing or canoes? In that case, it makes perfect sense to expect that you would do better on the “harder”—even the “arcane”—school-related items that are built around the words, ideas, and texts that you have actually been taught.

shutterstock_57833590

Photo courtesy of Shutterstock.

Or, put another way: Assessments of student learning are neither fair nor valid unless they measure only the content and skills that students have actually been given the opportunity to learn. And the only way to do that, of course, is to know what they have been taught—that is, in the presence of a defined curriculum.

The Problem with Curriculum

There are some very good reasons why the United States, unlike most of the world’s highest-performing nations, has avoided adopting a national curriculum for all of these years. As David K. Cohen has noted:

For school systems around the world, the infrastructure commonly includes student curricula or curriculum frameworks, exams to assess students’ learning of the curricula, instruction that centers on teaching that curriculum, and teacher education that aims to help prospective teachers learn how to teach the curricula. The U.S. has had no such common and unifying infrastructure for schools, owing in part to fragmented government (including local control) and traditions of weak state guidance about curriculum and teacher education.

Another huge issue is that “curriculum” has become a catch-all that describes everything from general performance standards all the way to student texts with scripted daily lesson plans. Thus, in any given discussion about the role of curriculum in a well functioning school system, it is very likely that the discussants are actually talking past each other. This has led to many unintentionally amusing statementsresponses and counter-responses, as each “side” tries to clarify what it and others are actually trying to promote and/or oppose.

In terms of equity concerns, I think that E. D. Hirsch has it exactly right. That is, we need to make sure that every American student—regardless of economic, geographic, racial or ethnic background—is provided with a “coherent, cumulative, and content-specific core curriculum” (see here, but also hereherehere and here).

As Hirsch uses the term, the “curriculum” should provide enough guidance to teachers to ensure that what is taught will prepare students for the learning that comes next, while remaining flexible enough for teachers (or schools or districts) to decide for themselves which specific materials and instructional approaches best meet the needs of any particular set of students. He uses the term “core” to mean both that which is most important, which should be taught in common to all students, as well as that which is foundational to the more personalized courses of study that students may choose for themselves during their high-school years. Thus, Hirsch’s Core Knowledge Sequence, which covers pre-K to 8th grade, could also be described as a curriculum framework or syllabus—a coherent “outline of the subjects in a course of study.”

It is no accident that Hirsch’s theory of action also squares with a great deal of national and international research suggesting that schools with greater curricular and instructional coherence achieve greater improvement in student performance (herehere and here).

So what might this look like in practice? In a 2003 Educational Researcher article, Lisa Delpit has given a rationale for why schools need to provide all students with access to “the culture of power”:

In my work in dozens of successful classrooms, effective teachers of low-income students of color take every opportunity to introduce children to complex material. While children are learning to “decode,” teachers read complex information to children above their reading level and engage in discussions about the information and the advanced vocabulary they encounter. Students are involved in activities that use the information and vocabulary in both creative and analytical ways, and teachers help them create metaphors for the new knowledge that connects it to their real lives. Students memorize and dramatize material that involves advanced vocabulary and linguistic forms. Students are engaged in thematic units that are ongoing and repeat important domain knowledge and develop vocabulary through repeated oral use. Students are asked to explain what they have learned to others, thus solidifying new knowledge. Not only do the teachers and schools who are successful with low-income children practice these strategies, but some other researchers (Beck et al., 2002Hirsch, 2003Stahl, 1991Sternberg, 1987, to name but a few) have documented the efficacies of the strategies as well. Successful instruction is constant, rigorous, integrated across disciplines, connected to students’ lived cultures, connected to their intellectual legacies, engaging, and designed for critical thinking and problem solving that is useful beyond the classroom.

Will the new SAT—or, for that matter, the new Common Core State Standards, which David Coleman also had a large hand in crafting—lead us toward this vision of educational opportunity? That is yet to be seen, but I would have much more confidence in the outcome if each state department of education had begun with a focus on teaching to the new standards, rather than just testing them. Where are the rich curriculum resources and professional development opportunities that would allow this vision to take hold? And, failing this, what exactly is it that we propose to measure?

__________

* Paradoxically, although the data confirm the expected class-based differences in the use of test prep courses, it should be noted that “blacks and Hispanics are more likely than whites from comparable backgrounds to utilize test preparation. The black-white gap is especially pronounced in the use of high school courses, private courses and private tutors.” See here for more on this.

 

Federal Policy, Teacher Effects

by Lisa Hansel
March 5th, 2014

My objective today is to put words in a few prominent researchers’ mouths—or better yet, their paper, “Learning that Lasts: Unpacking Variation in Teachers’ Effects on Students’ Long-Term Knowledge.” Benjamin Master, Susanna Loeb, and James Wyckoff have posted online a “preliminary draft,” which no one is supposed to cite. Blogging, I assume with all online content, is fair game. This paper is terrifically important. I just want to see a draft that more fully discusses the many factors that contribute to teacher effects.

Let’s start with why this paper is worth your time: It’s a blockbuster for those worried about the negative consequences of annual high-stakes test-based teacher evaluations. Looking at the long-term impact of teachers with high value added, the researchers conclude:

Evaluation and accountability systems may incentive educators to focus excessively on short-term tested outcomes in ways that are not ultimately beneficial for students…. Collectively, this body of evidence demonstrates that teachers’ instructional practices can influence their short-term value-added performance in ways that do not correspond with long-term success for students…. Overall, our results demonstrate that teachers’ effects on students’ long-term skills can vary substantially and systematically, in ways that are not fully captured by short-term value-added measures of instructional quality.

We clearly need education policy to incentivize (or at least not impede) meaningful educational gains, so I hope policymakers will heed this research.

To increase the odds that they will heed it, the paper needs one quick little addition: more forceful acknowledgement that teachers’ effects are influenced by many factors. Several federal and state policies could be explored as means of positively influencing curriculum and instruction. This is not simply a teacher issue. It is a standards, curriculum, assessment, accountability, teacher preparation, professional development, leadership, and resource-allocation issue.

Is it really these researchers’ job to remind readers of the broader context? No. It’s just something that, given the importance of the issue, I’m hoping they’ll want to note.

Barack_Obama_through_a_magnifying_glass

When it comes to teacher effects, context matters.

Reading this paper, it’s easy to get swept up in thinking the teacher makes all the difference. For example, the more academic knowledge teachers have, the more they seem to infuse that in their instruction, to great effect:

The within-subject value-added persistence of ELA teachers who attended a more competitive undergraduate institution is significantly and substantially higher than that of teachers who attended a less competitive institution…. Differences in persistence are similarly large when comparing teachers whose SAT Verbal exam scores or LAST licensure exam scores are in the top third of the teacher distribution, in comparison to lower-scoring teachers. In both cases, higher scoring teachers show greater persistence…. It is notable that our teacher ability characteristics predict large differences in ELA teachers’ value-added persistence, even though they are not themselves correlated with teachers’ short-term value-added effects.

One might be tempted to think these direct teacher effects are simply teacher-quality issues. But nothing in education is so simple. Ask yourself: what’s likely to make a person with the potential to have lasting effects want to be a teacher for years to come? Rigorous standards and engaging curriculum, meaningful assessments that support instruction, accountability policies that don’t incentivize test prep, academically demanding preparation programs, tailored professional development, helpful leaders, etc.

Much of the paper is devoted to examining potential student- vs. teacher-level drivers of the variation in teachers’ long-term impact. That, obviously, is a key question—I just want to see more acknowledgement that the “teacher effects” are federal-, state-, district-, and school-policy effects. Here’s the heart of the research:

Observable student characteristics related to their socio-economic status or prior ability also predict substantial variation in their ELA teachers’ value-added persistence. The persistence of achievement gains coming from having an effective teacher is far lower for students who are eligible for free lunch, are black or Hispanic, or whose twice-lagged ELA achievement scores are below the mean…. These students may be receiving ELA instruction that is less focused on long-term knowledge, or they may be less skilled at acquiring or retaining long-term knowledge.

Ultimately, the researchers conclude the primary issue is likely “instruction that is less focused on long-term knowledge”:

We see evidence of the importance of instruction in the positive association between teachers’ academic ability and their contributions to students’ long-term knowledge. Even more compelling, we find that schools that serve more disadvantaged students or that hire fewer of these high-ability teachers have lower value-added persistence in ELA for all of their students. Students, regardless of their prior test performance, who attend schools with many low-performing students demonstrate lower persistence of the learning gains they achieve from having a high value-added teacher. The persistence in low-achieving schools is less than half the rate of that in other schools. These findings provide evidence that instructional quality is a key driver of the variation that we observe in value-added persistence, and that school-level curriculum or instructional norms may foster differences in instructional quality. Unfortunately, we are unable to directly observe the instructional practices of teachers or schools in our sample. However, in light of prior research on educators’ responses to high stakes accountability pressures … one plausible explanation for our findings could be that schools serving lower performing students systematically prioritize gains in short-term tested achievement in ways that detract from teachers’ focus on long-term knowledge generation.

As I’ve said, there’s a whole lot beyond “school-level curriculum or instructional norms” that “may foster differences in instructional quality.” The authors of this paper know that—and it’s certainly not their fault that many policymakers need to be reminded. But they do. And if more policymakers get the message that we have a multifaceted, highly complex problem to address, perhaps more desperately needed research dollars will be provided and more varied policies will be piloted.

World Studies Could Make a World of Difference

by Lisa Hansel
November 5th, 2013

In my last post, I tossed out a not-quite-baked idea for a new academic major for elementary teachers: World Studies. That major would ensure teachers have the broad range of knowledge they need to introduce our students to the world via literature, history, geography, science, mathematics, and the arts.

One point that is obvious to the Core Knowledge community—yet somehow shocking and mysterious to most of the education world—is that before we could decide what academics future teachers need to study in their prep programs, we need to decide what all elementary students must learn. Drum roll: We need a core K-5 curriculum.

 

Refusing to identify and teach essential knowledge has consequences. (Image courtesy of Shutterstock.)

 

Some people might think we kinda sorta have a core curriculum with the Common Core standards. In math, I would agree—the actual math that has to be mastered each year is specified. It’s far from a full-blown curriculum, but it does provide concrete guidance on what math to teach. However, the Common Core ELA standards are nearly content free. They indicate the reading, writing, listening, and speaking skills students need to develop, but they do not outline content to be taught grade by grade. Instead, the ELA standards call on schools to create content-rich curricula infused with nonfiction texts, thereby systematically building broad knowledge across academic subjects.

I am grateful that the Common Core ELA standards explain the benefits of building knowledge. In implementation, schools need to realize that when they forge ahead without any shared core of content for each grade, they miss out on the many benefits of a coherent educational system. I’ve written about the problems with student mobility; today I want to share a terrific, little-known article by education professor David Cohen. In “Learning to Teach Nothing in Particular,” he explains the massive leaps forward we could make in student and teacher evaluation if only we specified what students are supposed to know:

Because local control and weak government were the foundations of U.S. public education, most of our school systems never developed the common instruments that are found in many national school systems…. These include a common curriculum or curriculum frameworks, common examinations tied to the curriculum, teacher education grounded in learning to teach the curriculum that students are to learn, and a teaching force whose members succeeded in those curriculum-based exams as students, among other things. Teachers who work with such infrastructure have instruments that they can use to set academic tasks tied to curriculum and assessment. They have a common vocabulary with which they can work together to identify, investigate, discuss, and solve problems of teaching and learning. Hence, they can have professional knowledge and skill, held in common….

Because there is no common infrastructure for U.S. public education, it has developed several anomalous features. One of the most important concerns testing: because there is no common curriculum, it is impossible to devise tests that assess the extent of students’ mastery of that curriculum. So, even though we’ve been testing student learning for nearly 100 years, only isolated programs (such as Advanced Placement and International Baccalaureate) have tested whether students learned what they were supposed to have been taught. In the early 1900s, when E. L. Thorndike and his colleagues and students invented tests of students’ academic performance, they devised tests that were designed to be independent of any particular curriculum. Nonetheless, those tests, and more recently developed similar tests, were and are used to assess students’ progress in learning. That has to rank as one of the strangest creations in the history of education.

Teacher education is a second anomaly: absent a common curriculum, teachers-in-training could not learn how to teach it, let alone how to teach it well. Hence, teacher education consists of efforts to teach future teachers to teach no particular curriculum. This is very strange, since to teach is always to teach something, but the governance structure of U.S. education has long forbidden the specification of what that something would be. For the most part, teacher education has been accommodating: typically, teacher candidates are taught how to teach no particular version of their subjects. That arrangement creates no incentives for those training to be teachers to learn, relatively deeply, what they would teach, nor does it create incentives for teacher educators to learn how to help teacher candidates learn how to teach a particular curriculum well. Instead, it offers incentives for them to teach novices whatever the teacher educators think is interesting or important (which often is not related to what happens in schools) or to offer a generic sort of teacher education. Most teachers report that, after receiving a teaching degree, they arrived in schools with little or no capability to teach particular subjects….

Absent a common curriculum, common assessments, common measures of performance, and teacher education tied to these things, it will be terribly difficult to devise technically valid and educationally usable means to judge and act on teaching performance. Building a coherent educational system would be a large task, but not nearly as daunting as trying to solve our educational problems without building such a system. Without standards and measures of quality practice—grounded in linked curriculum, assessments, and teacher education—it will be impossible to build a knowledgeable occupation of teaching, and a knowledgeable occupation is the only durable solution to the problem of quality in teaching.

 

Promethean Plan: A Teacher on Fulfilling the Intent of the Common Core, Part 2

by Guest Blogger
August 15th, 2013

By Mark Anderson

Mark Anderson, who became a NYC Teaching Fellow after working in retail and hospitality management, now teaches at Jonas Bronck Academy in the Bronx. His writing on educational improvement has appeared in Gotham Schools, the Times Union, VIVA Teachers, and other venues. Anderson also creates educational videos, including one that summarizes this blog post on fulfilling the intent of the Common Core.

In part 1 of this three-part series, Anderson discusses why skills-based teaching should no longer be predominant in ELA. In part 3, he discusses the dangers of infantilizing teachers.

 

Prometheus statue, University of Minho

 

Mistake #2: Placing the burden of teaching literacy entirely on ELA

As I noted in my last post, I believe the Common Core standards open a window of opportunity for systematically building students’ knowledge as teachers shift from “just-right texts” to complex texts. Another potentially transformative shift of the Common Core standards is the acknowledgment that literacy extends across all content areas. This is explicitly recognized by the standards in two ways: 1) the inclusion of literacy standards for social studies, science, and technical subjects in grades 6 – 12; and 2) the demand for an increase in informational texts.

Under key design considerations in the introduction to the literacy standards, Common Core’s authors state that the inclusion of social studies, science, and technical subjects “reflects the unique, time-honored place of ELA teachers in developing students’ literacy skills while at the same time recognizing that teachers in other areas must have a role in this development as well” (bold added).

They furthermore point out that “because the ELA classroom must focus on literature (stories, drama, and poetry) as well as literary nonfiction, a great deal of informational reading in grades 6–12 must take place in other classes” (bold added).

Yet within schools, these points are all too easily ignored or misconstrued. ELA teachers are evaluated by the literacy tests that their students are required to take. One of the greatest frustrations of being an ELA teacher, in fact, is that we are tested on factors that are often beyond our control, such as our students’ domain-specific knowledge. It’s no wonder, then, that many ELA teachers resort to skills-based teaching, grimly attempting to boost test scores by bolstering superficial, isolated skills.

That domain-specific knowledge is essential to literacy is a point that has been already been made much more cogently by others—such as Daniel Willingham, E. D. Hirsch, and Robert Pondiscio—and that is apparent in research. In my personal experience, I frequently teach students who are quite familiar with the skill of “inferencing,” for example, yet display little ability to make an accurate inference.

During my first years of teaching at my former elementary school, we had noted from our students’ literacy assessment data that inferencing was a deficient skill across all tested grades. All of us set about diligently teaching the skill. After going through a cycle or two of grade-level team “inquiry” on this skill, something slowly became apparent to me: our students couldn’t make accurate inferences because they didn’t understand what they were reading. The problem wasn’t lack of inferencing skill, it was lack of knowledge. This is when I first realized that we were failing our students because we didn’t have a coherent curriculum. Forget inferencing. Before we could do inquiry on anything, we had to have a solid, structured curriculum in place to refer to so that we could align what we were teaching across our classrooms and grades, and therefore address gaps in students’ knowledge and skills.

In most elementary schools, ELA is given heavy prominence, often to the detriment of music, arts, social studies, and science, as ELA test scores weigh heavily on schools’ performance. Yet this establishes a demoralizing catch-22, in that the domain-specific knowledge necessary for reading comprehension is then unable to be acquired.

If the research foundation and intent of the Common Core—to build the broad knowledge that is essential to literacy—remains unrecognized, then a simple and devastating misunderstanding of Common Core’s emphasis on “informational” texts will occur: ELA will avoid most literature altogether and focus on disparate expository texts instead, leaving us back at square one—an utter lack of coherency or of a systematic accumulation of knowledge.

The burden for literacy cannot remain on the shoulders of ELA alone. Literature, including literary nonfiction, is essential for gaining an understanding of the world, but it must be backed by domain-specific knowledge in other content areas.

In elementary school, this means that administrators need to shift their focus from ELA to social studies, science, arts, and music, and ensure that 90-minute literacy blocks are used to build knowledge, not simply to conduct independent reading and writing. This can be done most strategically by selecting a coherent body of texts for teacher read-alouds and whole class or small group exploration. In middle and high school, this means that social studies, science, and technical content area teachers need to be on board with also being teachers of literacy, and must be trained on the selection and teaching of texts that will build content-specific knowledge.

At my middle school, my grade-level team began developing this understanding by exploring the Common Core standards together. We discovered that the expectation that students would be able to cite evidence, read and comprehend complex grade-level texts, and write arguments that exhibit logical reasoning and address counterclaims extended across ELA, social studies, and science. Not only that, we found that argumentative standards in literacy closely aligned with expectations for mathematical practice, in that students were expected to construct viable arguments and critique the reasoning of others. Here is the initial document my team created to review and compare these argumentative standards.

Such an exploration, however, is only a foot in the door. Now we must consider how we can share strategies for teaching close reading, what qualitative and quantitative methods we can use to select grade-level complex texts, and in what way we can align these strategies across departments and grades. Furthermore, this also requires a shift on the part of us ELA teachers: we must be now be willing to consider how the texts and content we teach will align and build on the content taught in other classrooms.

While such an undertaking may appear daunting at first, the opportunity to collaborate on interdisciplinary papers, projects, and tasks is invigorating both for teachers and for students. At the end of the last school year, my ELA department began working with our social studies department to consider how we could align our poetry units with their units. We discovered that all social studies units shared a common theme of warfare, so we began selecting poems on warfare that would build on this theme and extend and enrich student understanding of multiple perspectives on war. This ability to strategically build on student knowledge strengthened student engagement, as students were able dive deeper into poems such as “Night in Blue” by Brian Turner and “Dulce et Decorum Est” by Wilfred Owen by drawing upon their knowledge of the experience of soldiers in traumatic modern wars.

Here’s one short-term measure we could take to ensure that the burden for teaching literacy does not fall only on the shoulders of ELA teachers:

  • Common Core-aligned literacy assessments should hold all the teachers for a grade level accountable.

Wait, what? You read that right. Make all the teachers on a grade level accountable for student performance on literacy tests. It might sound crazy, and I’m sure it will complicate the pristine “value-added” formulas that have been cooked up to evaluate individual teachers, but it’s the most effective means to ensure that schools actualize the teaching of knowledge as the key to literacy. So long as the burden of accountability for literacy tests falls solely to the domain of ELA, then the teaching of literacy will fall solely on the backs of ELA teachers, and the other content areas will therefore continue to be treated as secondary as testing hysteria arises during the year.

In the schools I have worked in, this hysteria is the inevitable accompaniment to high stakes testing. Teachers, despite themselves, begin referencing “the test” as a raison d’être of lessons. During this run up to testing, roughly December through May, a school’s frenetic focus is on ELA and math, with extra weekend and afterschool sessions piled on to reinforce all those isolated skills for good measure.

But now imagine if literacy were acknowledged as a grade-level team’s main objective. All hands would be on deck to ensure that content—across all domains—would be systematically taught and reinforced. In other words, we’d be doing what we should have been doing all along.

Longer term measures we could take to ensure that teaching literacy extends across content areas:

  • Schedule time each week for grade-level teacher teams to meet and collaborate on curriculum and pedagogy.
  • Include a focus on selecting and teaching complex texts in content-specific teacher training.

 

The Common Core Tests in Language Arts Will Soon Be Coming to Your Child’s School. Tell Your Local Superintendent: “Don’t Worry. Students Will Ace Those Tests If They Learn History, Civics, Literature, Science, and the Fine Arts.”

by E. D. Hirsch, Jr.
August 5th, 2013

This post originally appeared on the Huffington Post on July 30, 2013.

 

“A giant risk, as I see it, in the implementation of Common Core is that it will spawn skills-centric curricula. Indeed, every Common Core ‘expert’ we hear from seems to be advocating this approach.” 

This comment from an able and experienced teacher is one among several similar ones that teachers have recently posted on the Core Knowledge blog. Their worry is also mine.

The success of the Common Core Standards in Language Arts, adopted by more than 40 states, is supremely important for many reasons, not least because of the recent intensification of income inequality. Student scores on language arts tests are the single most reliable academic predictors of later income. The new language arts standards of the Common Core represent an historic opportunity for beneficial change in American schools—if they are put into effect intelligently.

But if you look at the data in Amazon books, you will see that the bestselling books about the Common Core are “skills-centric” ones that claim to prepare teachers for the new language arts standards by advocating techniques for “close reading” and for mastering “text complexity” as though such skills were the main ones for understanding a text no matter how unfamiliar a student might be with the topic of the text. The fact is, though, that students’ ability to engage in “close reading” and to manage “text complexity” is highly dependent on their degree of familiarity with the topic of the text. And the average likelihood of their possessing the requisite degree of familiarity with the various topics they encounter in life or on tests will depend upon the breadth of their knowledge. No amount of practice exercises (which takes time away from knowledge-gaining) will foster wide knowledge. If students know a lot they’ll easily learn to be skilled in reading and writing. But if they know little they will perform poorly on language tests—and in life.

We need to learn from recent painful experience. The failure of No Child Left Behind in fostering advanced language ability can be traced to the skills-centric test-prepping that left little room for the systematic gaining of knowledge. Of course there is one facet of the skills-centric approach to reading that should be applauded, and which did improve under NCLB—the teaching of decoding. Learning to translate those symbols on the page into sounds and words is a skill that ought to be taught systematically between kindergarten and second grade, beginning with simple letter-sound correspondences and progressing step-by-step to complicated Greek-based spellings. More systematic instruction in phonics explains why test scores went up in the earliest grades under NCLB. But its neglect of knowledge building explains why student scores did not go up in later grades when tests emphasize comprehension.

Test anxiety was paradoxically the main reason that schools spent so much time on abstract skills like “comprehension strategies” and “inferencing.” My aim in this blog post addressed to parents is to explain why the best test prep for their child under the new Common Core standards will be a more systematic approach to imparting knowledge. My argument is simple: If understanding a text depends on some prior familiarity with the topic, then that will also be true of the passages on a language-arts test.

The more a student knows the better he or she will perform on any language-arts test—whether or not that test is said to be “aligned” with the new Common Core standards. If we take a step back from the details of the Common Core standards we can see why this claim necessarily must be true. Older language arts tests—such as Gates-Macginitie, the Iowa Test of Basic Skills, the Stanford 9, the Degrees of Reading Power, the National Assessment of Educational Progress, and the verbal sections of the Armed Forces Qualification Test—correlate well with each other, indicating that they are all accurately probing underlying competence in language. If the results of the Common Core tests are not strongly correlated with these well-validated ones, then the technical validity of the new tests would rightly be deemed unsatisfactory, and no state ought to adopt them. There is no reason to think that the top experts making the new Common Core tests are not well aware of this technical issue of correlation. It’s the schools, rather, that need to reconsider what will really prepare their students for the new tests—and a productive life.

One reason that the schools have been applying a skills-centric approach is that they have regarded reading as a uniform skill that develops in stages, rather than a highly variable skill which depends on a person’s topic knowledge. The schools cannot be blamed for this. The stage-by-stage conception of reading is the theory that even top experts held up to a few decades ago. The notion that any text on any topic at the right level would enhance reading ability has encouraged our schools to tolerate a topic-incoherent curriculum in language arts. This indifference to knowledge building is the chief reason the verbal scores of our school leavers have stayed flat and low.

Cognitive scientists have found, however, that a student’s average level of reading skill, which is reasonably accurately indicated by the standard tests, masks wide fluctuations depending on the test taker’s familiarity with the topic. That’s why reading tests typically use multiple passages on different topics—characteristically about ten—to try to capture that average. And even then, the passages are not random but have been filtered through the net of grade-level criteria like word rarity and sentence length. The whole system has conspired to make schools think that the topic knowledge is less important than “reading level.” But now we know that the topic of the passage is far more important than the level. The more students know about a topic, the further above their level they can read on that topic. This new understanding of reading ability demands nothing less than a revolution in language arts instruction, with less emphasis on technique and more emphasis on the systematic acquisition of knowledge.

The new Common Core standards have recognized this research finding. They state that these standards “do not—indeed, cannot—enumerate all or even most of the content that students should learn. The Standards must therefore be complemented by a well-developed, content-rich curriculum.” And they add: “Students can only gain this foundation when the curriculum is intentionally and coherently structured to develop rich content knowledge within and across grades.” Well said! And we have to take care that the schools and the experts hear and act on that truth. Parents and concerned citizens should make sure that they do.

If any school wants to see a model for what this means in actual practice, there are a couple of resources on the Core Knowledge Foundation’s website that should be useful. Here is a grade-by-grade content sequence for all subjects in preschool through eighth grade that is downloadable for free. This sequence takes into account the knowledge that is most needed and used in written language in the United States—imparting topic familiarity as well as deeper insight across the topics that are most enabling for written communication. When schools use this sequence to write a rigorous curriculum, their students do well on language tests. Second, here is an early reading program—preschool through third grade—that systematically brings many history, science, and literary topics into the language arts classroom in sufficient depth so that the student becomes familiar with them. This pre-k – 3 program will be downloadable for free soon. For now, here’s the list of topics, with each taking about 2-3 weeks to teach. Each has about 10-12 teacher read-alouds and related class discussions and extension activities.

The coming of the Common Core standards and tests need not be a new, harrowing imposition on already besieged schools. Rather they are an historic opportunity—a new slate on which schools can write either a topic-indifferent, fragmented curriculum similar to what has failed before, or a new, exciting and successful orientation to knowledge. That’s what the top experts – the cognitive scientists—are telling us, and it’s a message that all parents, educators, and concerned citizens need to act upon.

Why Reading Is the Tougher Nut

by Lisa Hansel
June 3rd, 2013

In the New York Times last week, Brett Peiser, chief executive officer of the Uncommon Schools network, asked some very important questions about why students struggle with reading comprehension: “Is it a vocabulary issue? A background knowledge issue? A sentence length issue? How dense is the text?” As Peiser noted, all of these are facets of comprehension that take time to improve—but the problem is both easier to understand and harder to address than most educators realize.

In response to the Times article, Mike Goldstein, founder of the Match Charter School, noted on his blog that “At Match, we too have always had larger math gains than English gains.  Not just on MCAS, also on SAT.” He continues:

Two years ago I tackled this topic, and Tom [Hoffman] commented:

One problem with our testing regime is that it makes math look like half of the goal of a school. Math as a discipline is constructed and taught differently than all others. If your school is constructed to optimize math instruction, you’re not half way there; you’re more like 1/5th of the way.

Good thought.  I don’t know of any schools where kids have, say, 4 English classes and one math class.  And if we didn’t have the phrase “English class” — if instead we only had the component parts called “Literature class,” “Non-fiction class,” “Writing class,” “Vocabulary Building and Grammar class,” and so forth, you could imagine a school designed that way, with 4X the amount of time devoted to English.  And then instead of one test for English, we’d have 4 exams and one math exams.

I’m not saying this is a good idea given other tradeoffs, I’m saying that might result in more equal sized gains.

Goldstein is a fan of E. D. Hirsch’s work, so I hope he’ll smile when he reads this: I know of schools that have “4 English classes and one math class.” Schools that teach the full Core Knowledge Sequence teach language arts, world and American history and geography, visual arts, music, science, and mathematics. If you trust the decades of research showing that the key to literacy is broad knowledge, then these schools have, in effect, five English classes and one math class. The difference between this and typical schooling is that Core Knowledge schools teach all of these things from kindergarten through eighth grade. They do it rigorously, with content-rich curricula that flesh out the Sequence with domain-based studies in which reading, discussing, and writing content happens throughout each day.

To put this back in Peiser’s terms, if you focus on knowledge, issues with vocabulary, long sentences, and dense texts will be resolved.

As E. D. Hirsch has explained, vocabulary is best learned not with vocabulary lists, but in context while studying specific domains of knowledge. Key terms may be explained as needed, but most vocabulary is acquired through multiple exposures in multiple contexts. I saw an example of this recently at P.S. 104, the Bays Water School in Queens, NY, which is doing a terrific job of bringing Core Knowledge Language Arts to life. Second graders had recently listened to, discussed, and written about a series of texts their teacher read aloud about leaders—like Susan B. Anthony and Martin Luther King, Jr.—who had fought for a cause. Along the way, they learned the word “injustice.” I happened to be sitting in when the teacher started reading aloud from Charlotte’s Web: “Fern was up at daylight, trying to rid the world of injustice. As a result, she now has a pig.” Hands flew into the air as the children were eager to relate what they already knew about injustice to this new example.

It may be fairly obvious that building knowledge also builds vocabulary and enables comprehension. But the connection between knowledge and comprehending dense texts with long sentences may not be as obvious. Recent research by Diana AryaElfrieda Hiebert, and P. David Pearson shines some light:

The present study was designed to address the question of whether lexical or syntactic factors exert greater influence on the comprehension of elementary science texts. Based on previous research on text accessibility, it was expected that syntactic and lexical complexity would each affect students’ performance on science texts, and that these two types of text complexity together would additionally impact student performance. In order to test this hypothesis, 16 texts that varied in syntactic and lexical complexity across four different topics were constructed. Students read texts that ranged in complexity, each from a different topic.

Contrary to our hypotheses, syntactic complexity did not explain variance in performance across any of the four topics….

Lexical complexity significantly influenced comprehension performance for texts on two of the four topics, Tree Frogs and Soil, but not for texts on Jelly Beans and Toothpaste. This finding was consistent across all participant groups, including ELLs.

In writing about this research, E. D. Hirsch noted that “These results are at odds with the notion that the usual measures of sentence structure (and/or length) and vocabulary are reliable ways to determine the ‘right’ reading level of a text for a child. On the other hand, their findings are consistent with other work in language study, as Arya, Hiebert, and Pearson were quick to point out…. Given enough familiarity with a topic, children are able to make correct guesses about words they have never seen before. They are also able to disentangle complex syntax if their topic familiarity enables them to grasp the gist of a text.”

So, for comprehension, the primary question is this: How can we ensure that students are familiar with a broad range of topics? You don’t need four English classes—you need a content-rich, carefully organized, grade-by-grade curriculum. Introduce young children to the world through literature, science, history, geography, and the arts. Even in middle school, read to them texts that are too challenging for them to read themselves. Engage them in discussions and substantive writing every day.

And be patient.

Patience is the hard part. Our high-stakes accountability systems not only expect yearly gains, they reward bad practices. While drilling students in comprehension strategies can get a bump in scores, it will not lead to meaningful increases in literacy. Building broad knowledge (and thus broad vocabulary and the capacity to grasp complex text) takes time. With a content-rich, sequential curriculum, schools can build the necessary knowledge over time. What they can’t do is show big yearly gains on tests that are not matched to their curriculum (which is a hazard of all state tests in which the content of the passages is not revealed and thus can’t be studied).

Mike Petrilli of the Fordham Institute has proposed an interesting solution: a two-part accountability system that allows schools to create their own measures. Here’s his pitch (Duncan, Congress—are you listening?):

    1. First, as the default system, we keep something like we have today, but with better standards and tests. (Yes, common-core standards and tests.) Students are tested annually; schools are held accountable for making solid progress from September to June, with greater progress expected for students who are further behind. States and districts give these schools lots of assistance—with curriculum development, teacher training, and the like. Such a default system won’t lead to widespread excellence, but it will continue to raise the floor so that the “typical” school in America becomes better than it is today. (NB: I’d scrap any state-prescribed “accountability” below the level of the school. In other words, no more rigid teacher evaluation systems; leave personnel issues to the principals.) And it would provide taxpayers an assurance that they are getting a “public good” from their investment in public education (namely, a reasonably educated citizenry).
    2. Then we offer all public schools—district and charter—an opt-out alternative. They can propose to the state or its surrogate that they be held accountable to a different set of measures. My preferences would be those related to the long-term success of their graduates. School “inspections” could be part of the picture, too. These evaluation metrics would be rigorous, but designed to be supportive of, rather than oppositional to, the cause of excellent schools. And they might be particularly important to educators of a more progressive, anti-testing bent.

This plan would allow schools to focus on building knowledge instead of artificially boosting scores with drills on comprehension and test-taking strategies. Schools could commit to a strong curriculum, then measure progress in reading comprehension through curriculum-based tests of students’ growing knowledge of literature, science, history, geography, music, the arts. Such tests could involve reading, writing, and speaking to ensure that students are progressing in all aspects of language as they develop broad knowledge of the world.

Developing and teaching a content-rich, coherent curriculum is hard to do, but it’s also the only approach that works.

 

Happy 85th Birthday E. D. Hirsch, Part 4: Passing the Test

by Lisa Hansel
March 22nd, 2013

So far this week E. D. Hirsch has taught us that higher-order thinking depends on knowledge, that highly mobile students suffer acutely from our national refusal to establish a core of common content, and that there is an identifiable body of specific knowledge that facilitates communication. Now, on Hirsch’s birthday, we examine his game-changing policy prescription: curriculum-based reading tests.

Turning to pages 153 – 162 of Hirsch’s most recent book, The Making of Americans: Democracy and Our Schools, we learn “How to Ace a Reading Test.”*

Reading tests are attacked for cultural bias and other faults, but such complaints are unfounded. The tests are fast and accurate indexes of real-world reading ability. They correlate extremely well with one another and with actual capacity to learn and communicate. They consist, after all, of written passages, which students are to read and then answer questions on; that is, students are asked to exercise the very skill at issue…. The much more reasonable complaint is that an emphasis on testing has caused schools to devote too much time to drills and test preparation, with a consequent narrowing of the curriculum….

Yet the fault lies not with the tests but with the school administrators who have been persuaded that it is possible to drill for a reading test—on the mistaken assumption that reading is a skill like typing and that once you know the right techniques you can read any text addressed to a general audience. The bulk of time in early language-arts program today is spent practicing these abstract strategies on an incoherent array of uninformative fictions. The opportunity costs have been enormous. Schools are wasting hours upon hours practicing drills that are supposed to improve reading but that are actually depriving students of knowledge that could enhance their reading comprehension….

Here is the beginning of an actual passage from a New York State reading test for fourth grade:

There is a path that starts in Maine and ends in Georgia, 2,167 miles later. This path is called the Appalachian Trail. If you want, you can walk the whole way, although only some people who try to do this actually make it, because it is so far, and they get tired. The idea for the trail came from a man named Benton MacKaye. In 1921 he wrote an article about how people needed a nearby place where they could enjoy nature and take a break from work. He thought the Appalachian Mountains would be perfect for this.

The passage goes on for a while, and then come the questions. The first question, as usual, concerns the main idea:

This article is mostly about

1. how the Appalachian Trail came to exist.

2. when people can visit the Appalachian Trail.

3. who hikes the most on the Appalachian Trail.

4. why people work together on the Appalachian Trail.

Many educators see this question as probing the general skill of “finding the main idea.” It does not. Try to put yourself in the position of a disadvantaged fourth grader who knows nothing of hiking, does not know the difference between an Appalachian-type mountain and a Himalayan-type mountain, does not know where Maine and Georgia are, and does not grasp what it means to “enjoy nature.” Such a child, though much trained in comprehension strategies, might answer the question incorrectly. The student’s more advantaged counterpart, not innately smarter, just happens to be familiar with hiking in the Appalachians, has been to Maine and Georgia, and has had a lot of experience “enjoying nature.” The second student easily answers the various questions correctly. But not because he or she practiced comprehension strategies; this student has the background knowledge to comprehend what the passage is saying….

It has been shown decisively that subject-matter knowledge trumps formal skill in reading and that proficiency in one reading-comprehension task does not necessarily predict skill in another. Test makers implicitly acknowledge this by offering, in a typical reading test, as many as ten passages on varied topics. (If reading were a knowledge-independent skill, a single passage would suffice.)… Contrary to appearances and educators’ beliefs, these reading tests do not test comprehension strategies. There usually are questions like “What is the main idea of this passage?” but such a question probes ad hoc comprehension, not some general technique of finding the main idea. Reading comprehension is not a universal, repeatable skill like sounding out words or throwing a ball through a hoop. “Reading skill” is rather an overgeneralized abstraction that obscures what reading really is: an array of separate, content-constituted skills such as the ability to read about the Appalachian Mountains or the ability to read about the Civil War….

A reading test is inherently a knowledge test. Scoring well requires familiarity with the subjects of the test passages. Hence the tests are unfair to students who, through no fault of their own, have little general knowledge. Their homes have not provided it, and neither have the schools. This difference in knowledge, not any difference in ability, is the fundamental reason for the reading gap between white and minority students. We go to school for many years partly because it takes so long to build up the vast general knowledge and vocabulary we need to become mature readers.

Because this knowledge-gaining process is slow and cumulative, the type of general reading test now in use could be fair to all groups only above fifth or sixth grade, and only after good, coherent, content-based schooling in the previous grades. I therefore propose a policy change that would at one stroke raise reading scores and narrow the fairness gap. (As a side benefit, it would induce elementary schools to impart the general knowledge children need.) Let us institute curriculum-based reading tests in first, second, third, and fourth grades—that is to say, reading tests containing passages based on knowledge that children will have received directly from their schooling. In the early grades, when children are still gaining this knowledge slowly and in piecemeal fashion, it is impossible to give a fair test of any other sort….

We now have an answer to our question of how to enable all children to ace a reading test. We need to impart systematically—starting in the very earliest grades by reading aloud to students, then later in sequenced self-reading—the general knowledge that is taken for granted in writing addressed to a broad audience. If reading tests in early grades are based on a universe of pre-announced topics, general knowledge will assuredly be built up. By later grades, when the reading tests become the standard non-curriculum one, such as the NAEP tests, reading prowess will have risen dramatically.

Policy makers say they want to raise reading scores and narrow the fairness gap. But it seems doubtful that any state can now resist the anti-curriculum outcry that would result from actually putting curriculum-based testing into effect. Nonetheless, any state or district that courageously instituted knowledge- and curriculum-based early reading tests would see a very significant rise in students’ reading scores in later grades.

States would also see impressive results right away on the curriculum-based tests since the passages would be about content that all students had actually been taught. Just imagine: With curriculum-based tests, “test prep” would consist of studying literature, history, science, and the arts. Bringing that imaginary world to life relies on our leaders working together. So, this birthday retrospective ends with a call to the left and right, drawn from pages 186 – 187 of the Making of Americans.

One of the gravest disappointments I have felt in the twenty-fine years that I have been actively engaged in educational reform is the frustration of being warmly welcomed by conservatives but shunned by fellow liberals. The connection of the anti-curriculum movement with the Democratic Party is an accident of history, not a logical necessity. All the logic runs the other way. A dominant liberal aim is social justice, and a definite core curriculum in early grades is necessary to achieve it. Why should conservatives alone favor solid content while my fellow liberals buy into the rhetoric of the anti-curriculum theology that works against the liberal aims of community and equality? Practical improvement of our public education will require intellectual clarity and a depolarization of the issue. Left and right must get together on the principle of common content.

 

* For the endnotes, please refer to the book.

 

Do you have a birthday message for E. D. Hirsch or favorite quote from him? Please share it with all of us in the comments.

 

You may also be interested in other posts in this birthday retrospective:

Part 1: The Secret to Lifelong Learning

Part 2: Avoidable Injustice

Part 3: Breaking Free from the Siren Song

 

Blame the Tests

by E. D. Hirsch, Jr.
January 15th, 2013

In Praise of Samuel Messick 1931–1998, Part III

The chief practical impact of NCLB has been its principle of accountability. Adequate yearly progress, the law stated, must be determined by test scores in reading and math—not just for the school as a whole, but for key groups of students.

Now, a decade later, the result of the law, as many have complained, has been a narrowing of the school curriculum. In far too many schools,  the arts and humanities, and even science and civics, have been neglected—sacrificed on the altar of tests  without any substantial progress nationwide on the tests themselves. It is hard to decide whether to call NCLB a disaster or a catastrophe.

But I disagree with those who blame this failure on the accountability principle of NCLB. The law did not specify what tests in reading and math the schools were to use. If the states had responded with valid tests—defined by Messick as tests that are both accurate and have a productive effect on practice—the past decade would have seen much more progress.

Since NCLB, NAEP’s long-term trend assessment shows substantial increases in reading among the lowest-performing 9-year-olds—but nothing comparable in later grades. It also shows moderate increases in math among 9- and 13-year-olds.

So, it seems that a chief educational defect of the NCLB era lay in the later-grades reading tests; they simply do not have the same educational validity of the tests in early grades reading and in early- and middle-grades math.

 ****

It’s not very hard to make a verbal test that predicts how well a person will be able to read. One accurate method used by the military is the two-part verbal section of the multiple-choice Armed Forces Qualification Test (AFQT), which is known for its success in accurately predicting real-world competence. One section of the AFQT Verbal consists of 15 items based on short paragraphs on different subjects and in different styles to be completed in 13 minutes.  The other section of the AFQT Verbal is a vocabulary test with 35 items to be completed in 11 minutes. This 24-minute test predicts as well as any verbal test the range of your verbal abilities, your probable job competence and your future income level. It is a short, cheap and technically valid test. Some version of it could even serve as a school-leaving test.

Educators would certainly protest if that were done—if only because such a test would give very little guidance for classroom practice or curriculum. And this is the nub of the defects in the reading tests used during the era of NCLB: They did not adequately support curriculum and classroom practice. The tests in early-grades reading and in early- and middle-grades math did a better job of inducing productive classroom practice, and their results show it.

Early-grades reading tests, as Joseph Torgesen and his colleagues showed, probe chiefly phonics and fluency, not comprehension. Schools are now aware that students will be tested on phonics and fluency in early grades. In fact, these crucial early reading skills are among the few topics for which recent (pre-Common Core) state standards had begun to be highly specific. These more successful early reading tests were thus different from later ones in a critical respect:  They actually tested what students were supposed to be taught.

Hence in early reading, to its credit, NCLB induced a much greater correlation than before between standards, curriculum, teaching and tests. The tests became more valid in practice because they induced teachers to teach to a test based on a highly specific subject matter—phonics and fluency. Educators and policymakers recognized that teaching swift decoding was essential in the early grades, tests assessed swift decoding, and—mirabile dictu—there was an uptick in scores on those tests.

Since the improvements were impressive, let’s take a look at what has happened in over the past decade among the lowest performing 9-year-olds on NAEP’s long-term trend assessment in reading.

Note that there is little to no growth among higher-performing 9-year-olds, presumably because they had already mastered phonics and fluency.

Similarly, early- and middle-grades math tests probed substantive grade-by-grade math knowledge, as the state standards had become ever more specific in math. You can see where I’m going: Early reading and math improved because teachers typically teach to the tests (especially under NCLB-type accountability pressures), and the subject matter of these tests began to be more and more defined and predictable, causing a collaboration and reinforcement between tests and classroom practice.

In later-grades reading tests, where we have failed to improve, the tests have not been based on any clear, specific subject matter, so it has been impossible to teach to the tests in a productive way. (The lack of alignment between math course taking and the NAEP math assessment for 17-year-olds is similarly problematic.) Of course, there are many reasons why achievement might not rise. But specific subject matter, both taught and tested, is a necessary—if not sufficient—condition for test scores to rise.

In the absence of any specific subject matter for language arts, teachers, textbook makers, and test makers have conceived of reading comprehension as a strategy rather than as a side effect of broad knowledge. This inadequate strategy approach to language arts is reflected in the tests themselves. I have read many of them.  An inevitable question is something like this: “The main idea of this passage is….” And the theory behind such a question is that what is being tested is the ability of the student to strategize the meaning by “questioning the author” and performing other puzzle-solving techniques to get the right answer. But, as readers of this blog know, that is not what is being tested. The subject matter of the passage is.

This mistaken strategy-focused structure has made these tests not only valueless educationally, but worse—positively harmful. Such tests send out the misleading message that reading comprehension is chiefly strategizing. That idea has dominated language arts instruction in the past decade, which means that a great deal of time has been misspent on fruitless test-taking activities. Tragically, that time could have been spent on science, humanities and the arts—subjects that would have actually increased reading abilities (and been far more interesting).

The only way that later-grades reading tests can be made educationally valid is by adopting the more successful structure followed in early reading and math. An educationally valid test must be based on the specific substance that is taught at the grade level being tested (possibly with some sampling of specifics from previous and later grades for remediation and acceleration purposes). Testing what has been taught is the only way to foster collaboration and reinforcement between tests and classroom practice. An educationally valid reading test requires a specific curriculum—a subject of further conversations, no doubt.