Pass Christian Turns Lemons into Limoncello

by Lisa Hansel
November 19th, 2014

Tiny Pass Christian School District on Mississippi’s Gulf Coast has long been high performing. Despite two-thirds of its students qualifying for free or reduced-price lunch, Pass Christian has been in the top 1% of Mississippi districts for the last five years. Between 2005 and 2013, three of its four schools won Blue Ribbons—and last year, the Education Trust awarded Pass Christian High School a Dispelling the Myth Award. But even all those honors are not enough. While grappling with how to implement the Common Core State Standards (CCSS), educators in Pass Christian realized that they could do even better if they worked together.

This story begins in tragedy. Hurricane Katrina destroyed or damaged all of Pass Christian’s schools. Less than two months later, the schools reopened—piled up together in trailers on the least-damaged campus. Everyone had banded together to reopen so quickly. Then they really got to know each other, and they saw the devastation as an opportunity to create a cohesive school system that would sustain districtwide collaboration.

PC before

One of Pass Christian’s schools after Katrina.

Over the past several years of working to implement the CCSS, they have taken that collaboration to new heights with common curriculum, common teacher-made rubrics and assessments, and cross-school observations and mentoring.

Three teachers—Leslie Leyser, a high school English teacher, James Ashley Phillips, a high school mathematics teacher, and Mary Cordray, an elementary teacher—became CCSS liaisons working with teachers in grades 3 – 12 to offer ongoing, in-classroom support. While they saw plenty of good instruction, they also saw different content and expectations. They realized there were gaps and repetitions in what students learned, as well as more- and less-demanding assignments, assessments, and grading practices.

All students had a decent opportunity to learn—but some had a better opportunity to learn than others.

In English language arts, the CCSS set forth the skills students needed to master, but did not offer enough detail to ensure teachers shared the same high expectations for what mastering each skill looks like. An even greater issue was that the CCSS offered almost no guidance as to what to teach.

In a week-long conference that sounds a whole lot like the conference in which the Core Knowledge Sequence was created, all of the grades 3 – 12 ELA teachers set forth what they were teaching and hashed out what they would teach. In their first pass, they found an overwhelming amount of Holocaust literature. That’s clearly important, but since it was being taught in multiple grades, students were bored—and missing opportunities to learn other important content. Figuring out who should keep their Holocaust units and who should let go of them was tough. The teachers agreed to use their social studies and science standards as guides, and social studies teachers joined them to map out the new, shared ELA curriculum. Today, many of the novels, plays, short stories, and poems taught complement the social studies content, immersing students in, for example, the literature and history of the Civil Rights Movement through coordinated units. Many writing assignments are also coordinated.

Since Pass Christian is already very high scoring, it will be interesting to see over the next few years if their scores can be bumped up even higher (especially since the state assessments are not curriculum-based tests). Cognitive science promises benefits to students’ comprehension since the new curriculum avoids gaps and repetitions.

But Leslie Leyser knows all the effort is paying off. Recently, one of her students had an interview at Vanderbilt University. The admissions officer made a casual reference to the Allegory of the Cave, which the student had studied thanks to Pass Christian’s intentionally broad, rich, rigorous, common curriculum. The student—who before did not realize that such references are made all the time—was not only relieved, but overjoyed to be in the know. It wasn’t just a fleeting moment; it was a marker that this kid from a small shrimping town fit in at an elite university.

PC after

Pass Christian’s beautiful new schools are a fitting tribute to this community’s hard work.

Killing Three Birds with One Stone

by Lisa Hansel
November 4th, 2014

The Fordham Institute’s Aaron Churchill has an interesting new post weighing the merits of state-mandated testing in science and social studies. He notes the cons—like the minimal added information on school quality given the high correlations between scores on science and reading tests—and the pros—like reversing the narrowing of the curriculum driven by the high-stakes emphasis on reading and math. Then he sets forth four options (and ultimately recommends his third option):

1.) Keep the status quo. This would ensure that social studies and science are tested, but in non-consecutive years (e.g., science in grades 5 and 8). Yet the status quo still does not compel schools to treat these subjects as equal partners with ELA and math.

2.) Eliminate testing in social studies and science. This approach would reduce the cost of testing in these areas, which gives us little new information about student achievement for school-quality purposes. However, this option would likely encourage even more focus on ELA and math and would require a waiver from federal statute which presently requires science testing at least once in elementary, middle, and high school.

3.) Increase testing in social studies and science to the same frequency as math and ELA (i.e., test these subjects annually in grades 3-8). This would balance schools’ incentives to treat each subject equally, but at the cost of more time and money. From an information perspective, although little additional information is yielded in terms of student proficiency, annual testing could help analysts construct growth (i.e., “value-added”) measures for these subjects.

4.) Decrease testing in math and ELA to non-consecutive grades to match the frequency of social studies and science (e.g., test math and ELA in grades 4 and 6, not consecutively in grades 3-8). This would also balance schools’ incentives to treat subjects equally, but at the cost of less information and accountability. It would also require federal action to grant Ohio relief from consecutive-year-testing mandates in math and ELA in grades 3–8, or more likely, a rewritten federal law that governs state accountability (No Child Left Behind).

I’d like to offer a fifth option that assesses science and social studies yet has fewer tests: Draw the topics for the reading comprehension tests from the science and social studies standards. This blog recently explored the many drawbacks of current reading comprehension tests. In short, they contain a random smattering of “common” topics and topics that ought to be taught in school, but since they are not tied to any specific content that we can be certain has been taught, they inevitably privilege students who have acquired broad knowledge (usually at home).

The only way to construct truly fair reading comprehension tests is to ensure that the passages are on topics that have been taught in school. Since states’ English language arts standards usually do not specify which books, poems, short stories, etc. to teach in each grade, ELA standards are a poor guide for test developers concerned with equity. But states’ science and social studies standards usually do specify some core content to be taught in each grade. The obvious path forward is to construct reading comprehension tests that assess language arts skills using the science and social studies content specified in the standards. After all, skills depend on relevant prior knowledge, so such tests would give a more accurate picture of schools’ impact on students’ language abilities than our current random-content tests. And for the cost and time of just one test, we would have a decent gauge of three subjects.

Even better would be to draw the topics for passages on reading comprehension tests from science, social studies, art, music, geography, and civics standards. Such tests would (1) induce schools to develop a broad, content-rich curriculum and support teacher collaboration, (2) reduce the impact of the home on students’ scores, (3) build the knowledge and vocabulary that is essential to literacy, and (4) be the foundation for an accountability system that requires fewer tests yet still ensures that standards are being met.


Reading tests with science and social studies content that had been taught would be more equitable and more interesting. Image courtesy of Shutterstock.

New Leaders in Literacy

by Lisa Hansel
October 22nd, 2014

It used to be that advocating for building broad knowledge with a content-rich curriculum in the early grades was a lonely enterprise. No more! Whether it’s the focus on the early word gap or the Common Core’s explanation of literacy or the moral universe bending toward justice, knowledge is finally getting its due.

New reports from the National School Boards Association (NSBA), the Education Commission of the States (ECS), and the National Association of Elementary School Principals (NAESP) emphasize knowledge as a prerequisite to skills. In deference to the nature of the blogosphere, I’ve arranged them from shortest to longest.

In a new blog post and report, NSBA highlights the importance of nonfiction reading. The post takes on three widespread myths about the Common Core: that the standards push fictional literature out of the curriculum, that nonfiction doesn’t help prepare students for college, and that nonfiction is boring. Lovers of history, science, art, music, geography, civics, and Core Knowledge already know these claims are preposterous, but the post is worth a quick read. Here’s my favorite nugget: “Beth Deniell of Kennesaw State observed that the critics of informational reading ‘seem not to have considered that the contextual information students need in order to understand a literary work arrives in non-literary texts.’”

NSBA’s report takes a more data-oriented approach, showing that US students and adults lag behind in information reading ability. It will be eye-opening to anyone who thinks that life-long literacy—the type the enables prosperity and civic engagement—can be built on fiction alone.

For those new to building knowledge and literacy from preschool through third grade, ECS’s report is a great place to start. It moves rapidly through key points on everything from access to preschool and kindergarten to educational quality and continuity to financing and governance, and it offers snapshots of advances made by various states. With a state-level policymaker focus, the report only touches lightly on curriculum, but it does hit on the necessity of carefully sequencing learning experiences:

When children engage in a coherent set of high-quality P-3 learning experiences, the “fade out” effect (i.e., the notion that early gains in learning disappear later in school) is greatly diminished. Aligning standards, curricula and assessments ensures that young children engage in the right sequence of learning experiences at the right time. Alignment also ensures children are working toward building the set of skills and knowledge they will need as they move from a high-quality preschool to a high-quality full-day kindergarten and the early elementary grades. (p. 16)


Happy reader courtesy of Shutterstock.

NAESP’s report is both the longest and most informative. It’s a real gem for preschool directors and elementary principals. The first two sections—on preschool to third grade continuity, curriculum, and instruction—are especially strong. A few highlights:

Longitudinal studies have shown that an integrated learning continuum for children from age three to grade three contributes to sustaining achievement gains made in prekindergarten programs. (p. 11)

Alignment of standards, instruction, assessment and professional development ensures that students enter each successive grade having the foundation and skills needed to succeed there. Such alignment can reduce unnecessary repetition in instruction and allow for coverage of more instructional topics. A successful Pre-K-3 learning community aligns standards with a sequenced, coherent curriculum that describes what should be taught in each grade and in each subject and makes clear what mastery of each subject means and how it looks. (p. 21)

Learning is cumulative: Early learning facilitates later learning, and children who already know something about a particular topic often have an easier time learning more about it….

Effective instructional leaders support two specific early reading abilities: decoding and comprehension. Decoding is the ability to identify the words on a page; comprehension is the ability to understand what those words mean…. Instructional leaders support teaching that builds comprehension through read-alouds in prekindergarten, kindergarten and first grade, which help children to build knowledge and vocabulary….

Effective Pre-K-3 instructional leaders know that to be successful in a variety of subjects in middle and high school, students also need to build a basis of prior knowledge in science, history, civics, the arts, physical education and social-emotional learning. (p. 22)

E­ffective principals … know that student engagement is essential and that significant learning happens through exploration and play, particularly in prekindergarten and kindergarten. Strategies used to ensure understanding of key content and concepts will, however, change as children progress from grade to grade. For instance, once children enter first and second grade, effective principals know that these strategies shift to more direct instruction, integrated into engaging and dynamic learning opportunities. (p. 23)

To each of these very strong reports, the one thing I would add is domain-based instruction. As the research appendix to the Common Core ELA and literacy standards states, “Word acquisition occurs up to four times faster … when students have become familiar with the domain of the discourse and encounter the word in different contexts…. Vocabulary development … occurs most effectively [when] domains become familiar to the student over several days or weeks” (Appendix A, p. 33). In essence, most vocabulary is not learned through vocab lists, dictionaries, and weekly quizzes. Those things can be useful, but the vast majority of words are learned through multiple exposures in multiple contexts.

The difference between domain-based instruction and widely used theme-based units is focus. While a theme might be friendship and cover everything from family members to pets to pen pals, a domain is much narrower, such as the solar system or early Asian civilizations. The benefit of the domain is that vocabulary and concepts are repeated, deepened, and expanded with a carefully selected set of texts and supporting activities. While a theme might offer a great variety of words and ideas, little is repeated often enough to be learned. A focused domain provides a more genuine opportunity to learn; students get the multiple contexts they need and teachers have several opportunities to differentiate instruction, allowing everyone to master the core concepts and vocabulary of the domain.

Ideally, all children would learn from a content-specific, domain-based, cumulative curriculum that begins in preschool and extends through elementary school. When the preschool is located in the elementary school, collaboration on curriculum is feasible. But coordinating among a disparate set of child care settings, preschool centers, and elementary schools can be next to impossible. When planning together is unlikely, the next-best option is a preschool through fifth-grade program that ensures one grade builds on the next even without teachers interacting. A coherent program can provide continuity in developing language skills, vocabulary, and broad knowledge even as it shifts from a play-oriented approach in preschool to a more academic approach in the upper elementary grades. (Interested? Give Core Knowledge Language Arts a try. Preschool through third grade can be downloaded for free, and several units from grades 4 and 5 are also now freely available.)

Smarter Balanced Confuses Fairness and Validity

by Lisa Hansel
October 15th, 2014

Over the past two weeks, we’ve looked the ETS guidelines for fair assessments that PARCC adopted, as well as a sample item from PARCC. Now let’s turn to the “Bias and Sensitivity Guidelines” ETS developed for Smarter Balanced. While I can’t say that ETS’s guidelines for Smarter Balanced contradict those adopted by PARCC, they are different.

In the introduction, validity and fairness are equated: “if an item were intended to measure the ability to comprehend a reading passage in English, score differences between groups based on real differences in comprehension of English would be valid and, therefore, fair…. Fairness does not require that all groups have the same average scores. Fairness requires any existing differences in scores to be valid” (p. 6).

By this logic, since youth from higher-income homes, on average, have more academic and common knowledge than youth from lower-income homes, the test that conflates reading comprehension ability with opportunity to learn is perfectly fair. Valid I can agree with. Fair I cannot.

A couple pages later, further explanation is offered (p. 8):

Exposure to information

Stimuli for English language arts items have to be about some topic…. Which topics and contexts are fair to include in the Smarter Balanced assessments? One fairness concern is that students differ in exposure to information through their life experiences outside of school. For example, some students experience snow every winter, and some have never experienced snow. Some students swim in the ocean every summer, and some have never seen an ocean. Some students live in houses, some live in apartments, some live in mobile homes, and some are homeless.

Even though curricula differ, the concepts to which students are exposed in school tend to be much more similar than are their life experiences outside of school. If students have become familiar with concepts through exposure to them in the classroom, the use of those concepts as topics and contexts in test materials is fair, even if some students have not been exposed to the concepts through their life experiences. For example, a student in grade 4 should know what an ocean is through classroom exposure to the concept, even if he or she has never actually seen an ocean. A student does not have to live in a house to know what a house is, if there has been classroom exposure to the term. Similarly, a student does not have to be able to run in a race to know what a race is. Mention of snow does not make an item unacceptable for students living in warmer parts of the country if they have been exposed to the concept of snow in school.

Let’s pause here: “Even though curricula differ, the concepts to which students are exposed in school tend to be much more similar than are their life experiences outside of school.” Maybe. Maybe not.

It might be the case that all elementary schools teach snow, oceans, houses, races, and deserts. But does Smarter Balanced really test such banal topics? No. As far as I can tell from its sample items, practice tests, and activities for grades three to five, Smarter Balanced (like PARCC) tests a mix of common and not-so-common knowledge. Passages include Babe Ruth, recycling water in space, how gravity strengthens muscles, papermaking, the Tuskegee Airmen, tree frogs, murals, and much more.

The sample items strike me as comprehensible for third to fifth graders with broad knowledge, but I am highly skeptical that we can safely assume that children are acquiring such broad knowledge in their elementary schools.

As Ruth Wattenberg explained in “Complex Texts Require Complex Knowledge” (which was published in Fordham’s Knowledge at the Core: Don Hirsch, Core Knowledge, and the Future of the Common Core), students in the elementary grades have minimal opportunities to acquire knowledge in history and science. Reviews of basal readers in 1983 and 2003 revealed that they contained very little content. This would be a lost opportunity, not a serious problem, but for the fact that elementary schools tend to devote a substantial amounts of time to ELA instruction, and very little to social studies and science instruction. Wattenberg’s table (p. 35) should be shocking:

Grade and subject 1977 2000 2012
K–3 social studies 21 21 16
4–6 social studies 34 33 21
K–3 science 17 23 19
4–6 science 28 31 24

Even worse, Wattenberg found that “When elementary teachers were asked during what time period struggling students received extra instruction in ELA or math, 60 percent said that they were pulled from social studies class; 55 percent said from science class.”

In their home environments, the schools they attend, and the curriculum to which they are exposed, lower-income children do not have an equal opportunity to learn. As Smarter Balanced guidelines state, the assessment is fair “if students have become familiar with concepts through exposure to them in the classroom.” That’s a big if.

Making matters worse, Smarter Balanced (like PARCC) asserts that it’s just fine for some kids to have to learn during the test. Returning to the “Bias and Sensitivity Guidelines” (p. 8):

Information in the stimulus

A major purpose of reading is to learn about new things. Therefore, it is fair to include material that may be unfamiliar to students if the information necessary to answer the items is included in the tested material. For example, it is fair to test the ability of a student who has never been in a desert to comprehend an appropriate reading passage about a desert, as long as the information about deserts needed to respond to the items is found in the passage.

Last week, we explored how difficult it is to learn from one passage and how greatly such test items advantage students who already know the content that the passage is purportedly teaching. Smarter Balanced clearly disagrees with me. Here’s the introduction it its fourth grade Animal World activity:

The Classroom Activity introduces students to the context of a performance task, so they are not disadvantaged in demonstrating the skills the task intends to assess. Contextual elements include: an understanding of the setting or situation in which the task is placed, potentially unfamiliar concepts that are associated with the scenario; and key terms or vocabulary students will need to understand in order to meaningfully engage with and complete the performance task.

Please take a look at the activity—it assumes an enormous amount of knowledge. Even if it did not, the notion of learning and immediately demonstrating ability flies in the face of well-established research on human’s limited working memory capacity. There’s no getting around it: the students with relevant prior knowledge have a huge advantage.

One (sort of) positive note: I am cautiously optimistic that Smarter Balanced’s computer adaptive testing will help—a little. Here’s how it’s described:

Based on student responses, the computer program adjusts the difficulty of questions throughout the assessment. For example, a student who answers a question correctly will receive a more challenging item, while an incorrect answer generates an easier question. By adapting to the student as the assessment is taking place, these assessments present an individually tailored set of questions to each student and can quickly identify which skills students have mastered…. providing more accurate scores for all students across the full range of the achievement continuum.

In a hierarchical subject like math, the benefits of this adaptation are obvious. In reading, adaptation might help, but it might be misleading. Once a student has mastered decoding, what makes one passage “easier” to comprehend than another is driven primarily by the topic. If the student knows a lot about the topic, then factors like rare vocabulary (which isn’t rare to the reader with the relevant knowledge) and complex sentence structure are of little import. If a student does not know about the topic, then making the vocabulary and sentence structure easier will only help a little. The main way in which adaptive testing might be helpful is in varying the topics; “easier” passages would consist of more common topics, while more “challenging” passages would consist of less common, more academic topics. Then, if we examined the results carefully, we might see that a child lacks essential—teachable—academic knowledge.

Yet, I am only cautiously optimistic because the knowledge that drives reading comprehension is accumulated more haphazardly than hierarchically. One can have some academic knowledge while missing some common knowledge. A student whose grandparents lived most of their lives in Greece may know a great deal about ancient and modern Greece and be ready for a highly sophisticated passage comparing and contrasting ancient and modern Greece. That same student may have no knowledge of China, gravity, Harlem’s Jazz age, or other topics that might appear on the test. Without assessing topics that have been taught, I see no way to truly gauge a students’ comprehension ability (or what the teacher or school has added).

To reinforce the most important message—that comprehension depends on knowledge, and thus schools must systematically build knowledge—the tests need to be tied to the content taught or the high stakes need to be removed so schools will no longer take time out of regular instruction for test preparation.

PARCC Demonstrates the Benefits of Broad Knowledge

by Lisa Hansel
October 8th, 2014

Last week I explored the “ETS Guidelines for Fairness Review of Assessments.” These guidelines were adopted by PARCC, so I decided to take a look at PARCC’s sample items for English language arts. (PARCC is one of the two consortia of states with massive federal grants to create assessments aligned with the Common Core State Standards. Smarter Balanced is the other consortium; ETS developed somewhat different guidelines for it—I’ll take a look at those next week).

The knowledge demands in PARCC’s sample items are very broad, from cougars to Amelia Earhart to DNA testing. While I am happy to see some substantive questions—and hopeful that such test items will reinforce the standards’ call for systematically building knowledge with content-rich curriculum—I worry about the fairness of these assessments given how they are being used.

As I mentioned last week, it would be perfectly fair to have test passages on topics that had been taught. But, since the test developers do not know which topics are taught in each grade, they have to assess “common” knowledge. Due to well-documented differences in opportunities to learn at home and at school, some children know a good bit more common knowledge than others.

Let’s take a look at one of PARCC’s sample items for third grade. Three questions are asked based on the 631-word passage “How Animals Live.” There’s a typical main-idea question paired with a supporting-evidence question, and then a narrower question that assesses the “skills of rereading carefully to find specific information and of applying the understanding of a text.” Here’s the first section of the passage:

What All Animals Need

Almost all animals need water, food, oxygen, and shelter to live.

Animals get water from drinking or eating food. They get food by eating plants or other animals.

Animals get oxygen from air or water. Many land animals breathe with lungs. Many water animals breathe with gills.

Animals need shelter. Some animals find or build shelter. Other animals grow hard shells to protect themselves.

Many words here are undefined: oxygen, shelter, lungs, and gills. Are these words common to all third graders? Probably not, but much of the content is likely familiar to the vast majority of third graders—and perhaps enough content is familiar for most third graders to grasp the section (if not every word). Nonetheless, children who have learned about oxygen, shelter, lungs, and gills start out with a big advantage. They are reading and comprehending more quickly (which is extremely important in a timed test), and they are comfortable as they move into the more difficult content in the rest of the passage.

Here is the second section, and the beginning of the third:

Ways Of Grouping Animals

Animals can be grouped by their traits. A trait is the way an animal looks or acts. Animals get traits from their parents. Traits can be used to group animals.

Animals with Backbones

Animals with backbones belong to one group. A vertebrate is an animal with a backbone. Vertebrates’ backbones grow as they get older. Fish, snakes, and cats are all vertebrates. Vertebrates can look very different.

Let’s ignore the stiff, unengaging style. What really concerns me is the delusion that it is fair for content to be learned and applied during a high-stakes assessment. (As I noted last week, I do not dispute that the assessment is valid and reliable, so my concerns are with accountability policies, not really with this type of assessment.)

Since a definition of trait is given, it’s clear that some significant portion of third graders is not expected to know that word. Now imagine this is the first time you’ve encountered trait and examine the text:

A trait is the way an animal looks or acts…. Traits can be used to group animals…. Vertebrates can look very different.

What is a third grader to make of this? Clearly, vertebrates are not grouped by how they look.

A trait is the way an animal looks or acts…. Fish, snakes, and cats are all vertebrates.

Clearly, vertebrates are not grouped by how they act. How is the backbone (which is not defined) a trait, since it does not seem at all related to how all these animals look or act?

Making matters worse, understanding trait is essential to correctly answering the main-idea and supporting-evidence questions.


Do these vertebrates look or act alike?
(Image courtesy of Shutterstock.)

The fact is, vocabulary is not learned by being given a definition. Definitions can be helpful, but they are always incomplete. Words are learned through multiple exposures in multiple contexts. Even with simple words, multiple contexts are necessary: What, exactly, makes hoagies and gyros and PB&Js all sandwiches? I can’t even attempt a concise answer—I just know a sandwich when I see one.

Third graders who have had a unit on vertebrates and invertebrates will breeze through this passage; its inadequate definition of trait won’t matter. But students relying on this definition will surely be at least a little confused, possibly totally lost. The assessment will accurately tell us that children without knowledge of traits have limited comprehension of this passage—but it will not accurately tell us anything about their teachers or schools, for no one alerted the educators that the test would measure knowledge of traits.

Reading Test Developers Call Knowledge a Source of Bias

by Lisa Hansel
October 1st, 2014

You might expect to see a headline like this in the Onion, but you won’t. The Onion can’t run it because it isn’t just ironic—it’s 100% true.

A few years ago, a researcher at one of the big testing companies told me that when developing a reading comprehension test, knowledge is a source of bias. He did not mean the obvious stuff like knowledge of a yacht’s anemometer. He meant typical K–12 subject matter.

Since reading comprehension depends chiefly on knowledge of the topic (including the vocabulary) in the passage, the student with that knowledge has a large advantage over the student without it. And since there have always been great educational inequities in the United States, students’ knowledge—acquired both at home and at school—is very strongly correlated with socioeconomic status.

A logical solution would be to test reading comprehension using only those topics that students have been taught. Teachers can do this, but testing companies can’t—how would they have any idea what topics have been taught in each grade? It’s rare for districts, much less states, to indicate what or when specific books, people, ideas, and events should be taught.

Without a curriculum on which to base their assessments, testing companies have devised their own logic—which is sound given the bind they’re in. They distinguish between common and specialized knowledge, and then they select or write test passages that only have common knowledge. In essence, they’ve defined “reading comprehension skill” as including broad common knowledge. This is perfectly reasonable. When educators, parents, etc. think about reading comprehension ability, they do not think of the ability to read about trains or dolphins or lightning. They expect the ability to read about pretty much anything one encounters in daily life (including the news).

I already had this basic understanding, but still I found the “ETS Guidelines for Fairness Review of Assessments” eye opening. Guideline 1 is to “avoid cognitive sources of construct-irrelevant variance…. If construct-irrelevant knowledge or skill is required to answer an item and the knowledge or skill is not equally distributed across groups, then the fairness of the item is diminished” (p. 8). It continues, growing murkier:

Avoid unnecessarily difficult language. Use the most accessible level of language that is consistent with valid measurement…. Difficult words and language structures may be used if they are important for validity. For example, difficult words may be appropriate if the purpose of the test is to measure depth of general vocabulary or specialized terminology within a subject-matter area. It may be appropriate to use a difficult word if the word is defined in the test or its meaning is made clear by context. Complicated language structures may be appropriate if the purpose of the test is to measure the ability to read challenging material.

Avoid unnecessarily specialized vocabulary unless such vocabulary is important to the construct being assessed. What is considered unnecessarily specialized requires judgment. Take into account the maturity and educational level of the test takers in deciding which words are too specialized.

On page 10, it offers this handy table that “provides examples of common words that are generally acceptable and examples of specialized words that should be avoided…. The words are within several content areas known to be likely sources of construct-irrelevant knowledge”:

ETS table 1

Since having good reading comprehension means being able to read about a wide variety of common topics, table 1 seems just fine. But testing companies’ silence about what their reading comprehension tests actually measure is not. They say they are measuring “reading comprehension skill,” but their guidelines show that they are measuring a vaguely defined body of “common knowledge.”

Common words are not common to all. Even “common” knowledge is knowledge that must be taught, and right now—at home and at school—far too many children from low-income homes don’t have an opportunity to learn that knowledge (which is common to youth from middle-class and wealthy homes). That’s why reading comprehension scores are so strongly and stubbornly correlated with socioeconomic status.

These tests of “common” knowledge are accurate assessments and predictors of reading comprehension ability; but they are not fair or productive tests for holding children (and their teachers) accountable before an opportunity to learn has been provided.

If all testing companies would clearly explain that their reading comprehension tests are tests of knowledge, and if they would explain—as the ACT’s Chrys Dougherty does—that the only way to prepare for them is to build broad knowledge, then we could begin to create a fair and productive assessment and accountability system. Before the end of high school, all students should have broad enough knowledge to perform well on a reading comprehension test. But what about in third, fourth, or even seventh grade? In the early and middle grades, is a test drawn only from topics that have been taught in school the only fair way to test reading comprehension? How many years of systematically teaching “common” knowledge are needed before a reading comprehension test that is not tied to the curriculum is fair, especially for a student whose opportunities to learn outside of school are minimal?

The answer depends not so much on the test as on what is done with the scores. If we accepted the fact that reading comprehension depends on broad knowledge, we would radically alter our accountability policies. Scores on “common knowledge” reading comprehension tests would be recognized as useful indicators of where students are in their journey toward broad knowledge—they would not be mistaken for indicators of teaching quality or children’s capacity. Instead of holding schools accountable for scores on tests with content that is not tied to the curriculum, we would hold them accountable for creating a content-rich, comprehensive, well-sequenced curriculum and delivering it in a manner that ensures equal opportunity to learn. To narrow the inevitable gaps caused by differences in out-of-school experiences, we would dramatically increase free weekend and summer enrichment opportunities (for toddlers to teenagers) in lower-income neighborhoods. (We would also address a range of health-related disparities, but that’s a topic for another day.)

In sum, reading comprehension really does rely on having a great deal of common knowledge, so our current reading comprehension tests really are valid and reliable. To make them fair and productive, children from lower-income families must be given an equal opportunity to learn the knowledge that is “common” to children from higher-income homes.


Reading is always a test of knowledge (image courtesy of Shutterstock).

Ending the Cultural Divide

by Lisa Hansel
September 24th, 2014

It is possible to enjoy NASCAR and polo, FX and PBS, Homer Simpson and Homer’s epics—but it is not always easy. As much as we Americans claim to value diversity and multiculturalism, we tend to shun the idea that one person can embrace and embody more than one culture.

This struggle, writes Vicki Madden (an instructional coach in New York City) in the New York Times, is a reason why many students from low-income homes do not complete college:

Once those from lower socioeconomic backgrounds arrive on campus, it’s often the subtler things, the signifiers of who they are and where they come from, that cause the most trouble, challenging their very identity, comfort and right to be on that campus. The more elite the school, the wider that gap. I remember struggling with references to things I’d never heard of, from Homer to the Social Register. I couldn’t read The New York Times—not because the words were too hard, but because I didn’t have enough knowledge of the world to follow the articles. Hardest was the awareness that my own experiences were not only undervalued but often mocked, used to indicate when someone was stupid or low-class: No one at Barnard ate Velveeta or had ever butchered a deer.

We know that Core Knowledge narrows the knowledge gap. Students in schools using the Core Knowledge Sequence have Homer covered, leaning about Winslow Homer in kindergarten and sixth grade, and reading The Iliad and The Odyssey (or adaptations of them) in sixth grade.

That alone should help students feel that they belong on their college campuses. Explaining how to butcher a deer may be a point of pride for those who can also weave Franz Liszt and Duke Ellington (both studied in the seventh grade via the Sequence) into their conversations. But let’s be honest; it’s the unique student who is resilient when faced with insidious, frequent reminders that grandpa’s fishing boat can’t get you to St. Martin.


Grandpa’s boat courtesy of Shutterstock.

Just as important as the academic challenge is the personal one students face when they find the strength to stay and earn their degrees. Madden relays her own experience, summing up that of her students:

In college, I read Richard Rodriguez’s memoir, “Hunger of Memory,” in which he depicts his alienation from his family because of his education, painting a picture of the scholarship boy returned home to face his parents and finding only silence. Being young, I didn’t understand, believing myself immune to the idea that any gain might entail a corresponding loss. I was keen to exchange my Western hardscrabble life for the chance to be a New York City middle-class museumgoer. I’ve paid a price in estrangement from my own people, but I was willing. Not every 18-year-old will make that same choice, especially when race is factored in as well as class.

As the income gap widens and hardens, changing class means a bigger difference between where you came from and where you are going. Teachers like me can help prepare students academically for college work. College counselors can help with the choices, the federal financial aid application and all the bureaucratic details. But how can we help our students prepare for the tug of war in their souls?

To this I’d like to add two more questions: Is there a way to prevent students from feeling such a tug of war? Educators in Core Knowledge schools—and any schools that focus on closing the knowledge gap—have your students experienced similar challenges?

Maybe it’s just my own very fortunate experience—I grew up in West Virginia, yet attended rigorous schools and had close friends at “regular” schools—but my hope is that these challenges are minimized by an educational environment that values home culture and academic culture from the very first day of school. In passing, I’ve heard of Core Knowledge schools that have book clubs and study groups for parents, as well as cultural festivals in which families share the rich, varied heritages that make up their school community. I’d love to learn more about such opportunities for whole families to grow together. Please submit a comment below, or email me at Thank you!

Buckling Down (Not Under): Thoughtfully Embracing the Common Core

by Lisa Hansel
September 18th, 2014

Over the past few years, the Common Core standards have been hailed and reviled. But do these standards merit such passion? For great schools—with rigorous curricula, collaborative educators, and supportive administrators—state standards make little difference. For schools that need to improve, the Common Core standards offer some useful guidance—but it takes far more than standards to provide a sound education.

What really matters is how the standards are interpreted and implemented. Amidst the cacophony of voices weighing in, I found three level-headed pieces this week.


One courtesy of Shutterstock.

Let’s start with what just might be the Common Core’s biggest bugaboo: close reading. It’s the perfect example of a pedagogical strategy that can be useful if used occasionally to focus students’ attention, and harmful if taken to extremes. Daniel Willingham of the University of Virginia (and Core Knowledge’s board) weighed in to remind us that reading without drawing on prior knowledge is not possible—nor is comprehension without sufficient knowledge:

Writers count on their audience to bring knowledge to bear on the text…. Things get [tricky] when it’s not plain to the reader that he lacks information that is important to understanding the text. Researchers Eli Gottlieb and Sam Wineburg offered a wonderful example…. They asked Clergy, scientists, and historians to read George Washington’s Thanksgiving Proclamation of 1789. Clergy and scientists focused on Washington’s invoking the “providence of Almighty God,” and other religious phrasing, with clergy applauding the Christian tone, and scientists upset by it. Historians, in contrast, focused on what the document did not say; it did not mention Jesus, nor salvation, nor Christianity. They saw the document as Washington’s self-conscious attempt to craft a statement that would be acceptable to the diversity of religions practiced in the United States.

Willingham concludes that attending to language and word choices and rereading make sense. But if students lack the knowledge needed to fill the inevitable gaps in language, then the text alone will not foster learning. (For yet more sage advice on close reading, see this post by Harvard’s Catherine E. Snow.)


Two courtesy of Shutterstock.

Next up is a brief report on exactly the kind of Common Core implementation I’d love to see all over the country: teachers working together, using the standards as a guide to create a more rigorous, knowledge-building curriculum. Last week, Robert Pondiscio (who is now at the Fordham Institute) had the pleasure of visiting Washoe County, Nevada, where Aaron Grossman is leading the homegrown Core Task Project. Obviously impressed, Pondiscio writes:

Those who see Common Core as a curricular monoculture, a boondoggle for publishers, or a violation of local control would do well to come to Reno. They’re doing it all without enriching an army of publishers or consultants—and with regional flair. Like children everywhere under Common Core, Reno kids will be expected to cite evidence in their reading and writing. But unlike anywhere else, they’ll be doing so by studying Nevada mining, conflict, and compromise among native tribes in Northern Nevada, and Nevada statehood and the state’s role in the Civil War—Common Core units developed by the Core Task Project. “We’re able to explain to the electorate that everything we’re doing is in-house, matched to our community values and to things we think are important,” Grossman observes.

Among the many resources the Washoe teachers are drawing on is Core Knowledge Language Arts. By thoughtfully crafting their curriculum, they have found that they have plenty of time for both the broad knowledge all students need and the knowledge of Nevada that they cherish.


Three courtesy of Shutterstock.

Lastly, in a wide-ranging podcast, Linda Darling-Hammond of Stanford offers sound advice on how the Common Core (or any college- and career-focused standards) can improve teacher and student learning. She also highlights research on assessments and suggestions for more productive accountability policies.

Curriculum Doesn’t Matter, Unless You Care about Achievement and Mobility

by Lisa Hansel
September 11th, 2014

Five years ago, Russ Whitehurst published an important paper comparing the effects of various education reforms. Better teachers and curriculum rose to the top, with what is taught being just as important as who is doing the teaching. But that finding didn’t fit with reformers’ obsession with teachers, so the paper was largely ignored. Two years ago, Whitehurst and Matthew Chingos did a more extensive look, confirming the previous findings and challenging states to begin gathering data on which materials are being used in schools.

Now, with pressure to interpret and meet the Common Core standards, curriculum, textbooks, and other instructional materials are finally getting wider attention. Much of that attention has been negative, as those who are against the standards seem to enjoy finding misinterpretations of the standards’ intent. I find that gotcha game silly; no one really expects initial stabs at Common Core–aligned materials to be terrific. Over time they will improve—and with support they will improve more quickly.

I’m thrilled to see growing interest in providing that support. The Helmsley Charitable Trust, the Hewlett Foundation, and the Bill & Melinda Gates Foundation are leading the way by funding several efforts, the most promising of which just might be Preparing to launch in the winter, is involving teachers in intensive reviews of K–12 math and ELA materials, and the reviews will be free online. At the same time, Rachel Leifer and Denis Udall, program officers with Helmsley and Hewlett (respectively), have penned a plea for “Disrupting the Textbook Status Quo.” Since Core Knowledge is a small non-profit trying to offer better materials for free online, I am heartened by their call for more philanthropies to support development and dissemination efforts so that “a marketplace for instructional materials that rewards quality and innovation” can be created.

For philanthropies that aren’t quite convinced that curriculum matters, here’s one more study to add to the great work noted by Leifer and Udall. In “The Role of the School Curriculum in Social Mobility,” Cristina Iannelli shows that the content of the curriculum has lasting effects.* This study is important not only because of its findings, but because relatively few studies look at the actual courses students take. Iannelli is a professor in the UK; she used the UK’s National Child Development Study (NCDS), which tracks all babies born in the UK in 1958, gathering data at ages 7, 11, 16, 23, 33, 42, 46, and 50. As Iannelli writes (p. 910):

The people in the study were in secondary schools between 1969 and 1976 during the period of reorganisation of British secondary education from a selective to a comprehensive system. The coexistence of different secondary school systems at the end of the 1960s provides an excellent opportunity to study the effect of studying different curricula and attending different types of school on individuals’ chances of reaching the highest social classes of destination.

Focusing on social and occupational status at ages 23, 33, and 42, Iannelli’s findings at age 23 were what you’d expect: parental education and school selectivity had a big impact. Curriculum did too, but what’s really interesting is that the relative importance of curriculum went up—and the importance of parental education and school selectivity went down—as people aged (p. 923–924):

Selective schools, languages, English, mathematics and science subjects had a positive and significant effect on the chances of being in the top social classes and reduced the chances of entering the bottom classes…. [The] results suggest that the indirect effects of parental education via school types and curricula are stronger at the beginning of respondents’ occupational career than at later stages. The opposite is true for social class of origin: it is in the long run that school and curricular choices emerge more powerfully as transmitters of social advantages….

We tested whether the effect of curriculum and school type at age 33 and 42 was simply a result of their effect at age 23 and 33…. The results, before and after controlling for prior occupational destinations, barely change in the analysis of class of destination at 33, indicating that the effect of studying different subjects and attending various types of schools continues beyond the point of career entry. However, when analysing destinations at age 42 after controlling for destination at age 33, while the effect of subjects remains the same the effect of school types reduces and is no longer significant. These results suggest that the subjects studied at school are very good predictors of individuals’ destinations at all three stages of occupational career. On the other hand, the school type attended has a significant short-term and medium-term effect on individuals’ occupational destinations but they become less important for explaining later destinations. This may indicate that cognitive effects may be more persistent than institutional status effects…. The long-lasting effects of some school subjects may indicate that they provide skills, such as critical thinking and complex reasoning, which are useful for individuals’ future occupational careers. [Emphasis added.]

Studying languages, English, mathematics, and science does indeed enhance critical thinking and complex reasoning abilities. In fact, these crucial abilities can only be increased by developing rich knowledge. Cognitive science on how knowledge builds on knowledge, and skills depend on knowledge, would predict these occupational findings. The students who took many courses in these subjects began their careers with a solid foundation of knowledge and skills that enabled them to learn and grow at work.

We won’t ever be able to predict where each young person’s career will go, but we have plenty of evidence as to the type of education that offers the best preparation: a broad, rich, academic curriculum that builds content knowledge and skills together.


Mobility by knowledge courtesy of Shutterstock.


* Many thanks to Webs of Substance for finding this study!

Pariah to Politico 50

by Lisa Hansel
September 4th, 2014

After 30 years of being misunderstood, E. D. Hirsch’s dedication to equalizing opportunity is being widely recognized. Today, he has the honor of being included in the Politico 50.

E D Hirsch

As noted in Politico’s encapsulation of his work, “Hirsch’s argument was revolutionary: All children, regardless of background, should be taught the shared intellectual foundation—from Euclid to Shakespeare to Seneca Falls—needed ‘to thrive in the modern world.’”

Thriving, regardless of the accident of birth, has always been his driving force. In her brilliant article, “‘I’ve Been a Pariah for So Long,’” Peg Tyre explores how Hirsch has been revered and reviled:

In 1978, between stints as head of UVA’s English Department, Hirsch was conducting research at a nearby community college. There, he observed that the largely African-American low-income students could read short works of narrative fiction but could barely wring meaning from a piece about Gen. Robert E. Lee’s surrender to Gen. Ulysses S. Grant at Appomattox because they lacked basic knowledge about the Civil War. “What I saw is that background knowledge really mattered,” Hirsch says….

To level the playing field between rich and poor, schools should intentionally build background knowledge in all children in a wide range of subjects, or, says Hirsch, “It will be impossible to break the cycle of illiteracy that persists from parent to child.”

Which is where his List came in. To make it, Hirsch, along with two professors, scanned newspapers and popular journals for high-frequency concepts and then polled 500 professionals—lawyers and writers—to determine which of those concepts were the most crucial for cultural literacy….

OK, I say, that might have made sense in the 1970s, but what about now when the complexion of American public schools is changing? … How can any dictionary of cultural literacy keep pace with such a rapidly changing world? Hirsch grows crisp. “Why do you think one’s color or ethnicity would affect one’s vocabulary? Without a doubt, Latino culture is having a big influence on America, and the language of culture will change around the margins. But educated conversation is still going on. And you want to make sure those kids—particularly those kids—have the tools they need to be included.”

Tyre concludes, “In the autumn of life, with three grown children (two of whom are teachers), he’ll take vindication where he can. ‘The point wasn’t to perpetuate the culture of power,’ he tells me. ‘It was to open the door to kids who don’t have the keys to power.’”