Smarter Balanced Confuses Fairness and Validity

by Lisa Hansel
October 15th, 2014

Over the past two weeks, we’ve looked the ETS guidelines for fair assessments that PARCC adopted, as well as a sample item from PARCC. Now let’s turn to the “Bias and Sensitivity Guidelines” ETS developed for Smarter Balanced. While I can’t say that ETS’s guidelines for Smarter Balanced contradict those adopted by PARCC, they are different.

In the introduction, validity and fairness are equated: “if an item were intended to measure the ability to comprehend a reading passage in English, score differences between groups based on real differences in comprehension of English would be valid and, therefore, fair…. Fairness does not require that all groups have the same average scores. Fairness requires any existing differences in scores to be valid” (p. 6).

By this logic, since youth from higher-income homes, on average, have more academic and common knowledge than youth from lower-income homes, the test that conflates reading comprehension ability with opportunity to learn is perfectly fair. Valid I can agree with. Fair I cannot.

A couple pages later, further explanation is offered (p. 8):

Exposure to information

Stimuli for English language arts items have to be about some topic…. Which topics and contexts are fair to include in the Smarter Balanced assessments? One fairness concern is that students differ in exposure to information through their life experiences outside of school. For example, some students experience snow every winter, and some have never experienced snow. Some students swim in the ocean every summer, and some have never seen an ocean. Some students live in houses, some live in apartments, some live in mobile homes, and some are homeless.

Even though curricula differ, the concepts to which students are exposed in school tend to be much more similar than are their life experiences outside of school. If students have become familiar with concepts through exposure to them in the classroom, the use of those concepts as topics and contexts in test materials is fair, even if some students have not been exposed to the concepts through their life experiences. For example, a student in grade 4 should know what an ocean is through classroom exposure to the concept, even if he or she has never actually seen an ocean. A student does not have to live in a house to know what a house is, if there has been classroom exposure to the term. Similarly, a student does not have to be able to run in a race to know what a race is. Mention of snow does not make an item unacceptable for students living in warmer parts of the country if they have been exposed to the concept of snow in school.

Let’s pause here: “Even though curricula differ, the concepts to which students are exposed in school tend to be much more similar than are their life experiences outside of school.” Maybe. Maybe not.

It might be the case that all elementary schools teach snow, oceans, houses, races, and deserts. But does Smarter Balanced really test such banal topics? No. As far as I can tell from its sample items, practice tests, and activities for grades three to five, Smarter Balanced (like PARCC) tests a mix of common and not-so-common knowledge. Passages include Babe Ruth, recycling water in space, how gravity strengthens muscles, papermaking, the Tuskegee Airmen, tree frogs, murals, and much more.

The sample items strike me as comprehensible for third to fifth graders with broad knowledge, but I am highly skeptical that we can safely assume that children are acquiring such broad knowledge in their elementary schools.

As Ruth Wattenberg explained in “Complex Texts Require Complex Knowledge” (which was published in Fordham’s Knowledge at the Core: Don Hirsch, Core Knowledge, and the Future of the Common Core), students in the elementary grades have minimal opportunities to acquire knowledge in history and science. Reviews of basal readers in 1983 and 2003 revealed that they contained very little content. This would be a lost opportunity, not a serious problem, but for the fact that elementary schools tend to devote a substantial amounts of time to ELA instruction, and very little to social studies and science instruction. Wattenberg’s table (p. 35) should be shocking:

Grade and subject 1977 2000 2012
K–3 social studies 21 21 16
4–6 social studies 34 33 21
K–3 science 17 23 19
4–6 science 28 31 24

Even worse, Wattenberg found that “When elementary teachers were asked during what time period struggling students received extra instruction in ELA or math, 60 percent said that they were pulled from social studies class; 55 percent said from science class.”

In their home environments, the schools they attend, and the curriculum to which they are exposed, lower-income children do not have an equal opportunity to learn. As Smarter Balanced guidelines state, the assessment is fair “if students have become familiar with concepts through exposure to them in the classroom.” That’s a big if.

Making matters worse, Smarter Balanced (like PARCC) asserts that it’s just fine for some kids to have to learn during the test. Returning to the “Bias and Sensitivity Guidelines” (p. 8):

Information in the stimulus

A major purpose of reading is to learn about new things. Therefore, it is fair to include material that may be unfamiliar to students if the information necessary to answer the items is included in the tested material. For example, it is fair to test the ability of a student who has never been in a desert to comprehend an appropriate reading passage about a desert, as long as the information about deserts needed to respond to the items is found in the passage.

Last week, we explored how difficult it is to learn from one passage and how greatly such test items advantage students who already know the content that the passage is purportedly teaching. Smarter Balanced clearly disagrees with me. Here’s the introduction it its fourth grade Animal World activity:

The Classroom Activity introduces students to the context of a performance task, so they are not disadvantaged in demonstrating the skills the task intends to assess. Contextual elements include: an understanding of the setting or situation in which the task is placed, potentially unfamiliar concepts that are associated with the scenario; and key terms or vocabulary students will need to understand in order to meaningfully engage with and complete the performance task.

Please take a look at the activity—it assumes an enormous amount of knowledge. Even if it did not, the notion of learning and immediately demonstrating ability flies in the face of well-established research on human’s limited working memory capacity. There’s no getting around it: the students with relevant prior knowledge have a huge advantage.

One (sort of) positive note: I am cautiously optimistic that Smarter Balanced’s computer adaptive testing will help—a little. Here’s how it’s described:

Based on student responses, the computer program adjusts the difficulty of questions throughout the assessment. For example, a student who answers a question correctly will receive a more challenging item, while an incorrect answer generates an easier question. By adapting to the student as the assessment is taking place, these assessments present an individually tailored set of questions to each student and can quickly identify which skills students have mastered…. providing more accurate scores for all students across the full range of the achievement continuum.

In a hierarchical subject like math, the benefits of this adaptation are obvious. In reading, adaptation might help, but it might be misleading. Once a student has mastered decoding, what makes one passage “easier” to comprehend than another is driven primarily by the topic. If the student knows a lot about the topic, then factors like rare vocabulary (which isn’t rare to the reader with the relevant knowledge) and complex sentence structure are of little import. If a student does not know about the topic, then making the vocabulary and sentence structure easier will only help a little. The main way in which adaptive testing might be helpful is in varying the topics; “easier” passages would consist of more common topics, while more “challenging” passages would consist of less common, more academic topics. Then, if we examined the results carefully, we might see that a child lacks essential—teachable—academic knowledge.

Yet, I am only cautiously optimistic because the knowledge that drives reading comprehension is accumulated more haphazardly than hierarchically. One can have some academic knowledge while missing some common knowledge. A student whose grandparents lived most of their lives in Greece may know a great deal about ancient and modern Greece and be ready for a highly sophisticated passage comparing and contrasting ancient and modern Greece. That same student may have no knowledge of China, gravity, Harlem’s Jazz age, or other topics that might appear on the test. Without assessing topics that have been taught, I see no way to truly gauge a students’ comprehension ability (or what the teacher or school has added).

To reinforce the most important message—that comprehension depends on knowledge, and thus schools must systematically build knowledge—the tests need to be tied to the content taught or the high stakes need to be removed so schools will no longer take time out of regular instruction for test preparation.

Knowledge for What?

by Guest Blogger
August 28th, 2014

By Will Fitzhugh

Will Fitzhugh is the founder and editor of the Concord Review, a scholarly history journal with well-researched essays by high school students.

Education is an important issue these days, which is both good and bad. Good, because we need to pay more attention to the work of our schools these days, and not so good, because lots of people who know all about convertible debentures, initial public offerings, etc., think they must know a lot about teaching and learning as well.

There is prolonged debate about the role of education in promoting citizenship, character, lifelong learning (try living without learning sometime), career readiness, environmental awareness, respect for diversity, and on and on.

What I find missing most of the time is any suggestion that after an education (and during an education) it might be nice to have gained some knowledge. “How did so many countries and peoples get involved in World War I?” for example. “How did Jefferson feel when he had to change his mind about presidential prerogatives under the Constitution when the Louisiana territory came up for sale?” “What was the crucial insight that led Watson and Crick to the understanding of the double helix?”

When people raise the question of “Knowledge for What?” my response is usually: for its own sake. E. D. Hirsch and others have shown that having knowledge is what makes it possible to gain more knowledge. And being able to gain more knowledge is really necessary in life, I would agree. In addition, perhaps this is just my bias as an editor and publisher of interesting history research papers, I also feel that gaining knowledge is really one of the essential pleasures in life. Jefferson said: “I could not live without books.” I don’t think that was only because some books could aid him in the many architectural and agricultural innovations he cared about.

James Madison wrote: “Knowledge will forever govern ignorance; and a people who mean to be their own governors must arm themselves with the power which knowledge gives…. What spectacle can be more edifying or more seasonable, than that of liberty and learning, each leaning on the other for their mutual and surest support?”

I have been told that Jefferson may even have been able to play some of Mozart’s new work on his violin, and it seems likely he valued that, whether or not he could prove it made him a more efficient farmer, or a more productive President.

Sometimes, I would suggest, in our vigorous (frantic) pursuit of the practical, we skip over some of the things that are of the greatest (practical) value. Once Sir Alexander Fleming was given a tour of the brand-new gleaming headquarters of the Salk Institute by an eager young Ph.D. After the tour, the guide could not help but say: “Just think what you could have discovered if you had only had this state-of-the-art equipment!?!” And Sir Alexander Fleming said, perhaps kindly, “Not penicillin.”

So, by all means, let us introduce more computer technology, more vocational training, more college- and career-ready standards for critical thinking, textual analysis, deeper reading and all of that. But let’s also remember that one of the goals of education must be the acquisition of knowledge, including knowledge of history. We can never be completely sure, at the time we acquire it, when or in what ways some knowledge may be useful in itself in our brief lives as human beings.

shutterstock_92746891

Essential pleasures courtesy of Shutterstock.

 

If Only We Had Listened…

by Lisa Hansel
July 15th, 2014

Thanks to my history-loving father-in-law, I’m holding a perfectly preserved editorial from the 1948 Washington Times-Herald—Tuesday, February 24, 1948, to be exact. It’s self-explanatory, so here goes:

More About Schools

A few days ago, we shot a short editorial under the title “Something Wrong With Education.” The piece told how the New York State Department of Education, after an exhaustive survey, had estimated the only about 65% of high school juniors can spell everyday words such as “develop,” “meant,” “athletic,” etc.

From this we inferred that something was moldy in present-day public education methods, and that the something probably wasn’t traceable to either the teachers or the children.

A couple of mornings after that editorial was printed, three mothers of primary public school children in the first and second grades visited your correspondent. There ensued what seemed to us a most interesting conversation—interesting enough to boil down to its essential here. Let’s call the ladies Mrs. A, Mrs. B, and Mrs, C.

Mrs. A: “The editorial was all right, and I only wish you’d put it at the top of the column instead of the bottom. But the trouble doesn’t start in the high schools. It starts right down in the first grade.”

Mrs. B: “Which they’re turning into kindergarten, where the children don’t learn a thing. Likewise the second.”

Mrs. C: “They call it progressive education. Humph.”

Mrs. A: “Puppets.”

Mrs. C: “Yes, puppets. Puppets they want the children to make out of carrots and things. Even have a book called ‘Puppetry in the Classroom’ or something like that.”

Mrs. B: “It has diagrams—do this and do that, with letters A-B-C to show you what to do to make a puppet. But they don’t teach the children what letters are, or what they mean, or how to read, so how can they make head or tail of the diagrams?”

Mrs. A: “There’s a rule, too, against having any letters or figures on the blackboard. They claim a child of 6 can’t grasp those things and mustn’t be bothered with them, or his co-ordination will go bad—at least I think they call it co-ordination.”

Mrs. C: “Of course the fact is that a child at that age is as curious as can be, and loves to fool with pencils, and is usually just crazy to find out how to write like grownups, how to read the papers, how to count—”

Mrs. B: “Oh, yes, about counting. They don’t teach them nowadays to learn figures and add ‘em or subtract ‘em. Oh no—they’ve got to count beads on strings, or bounce rubber balls up and down. Ant they mustn’t learn to go above number 5 for a year or two, because that would strain their brains. Humph.”…

Mrs. C: “It’s not the teachers’ fault. I’m sure of that. Plenty of them will tell you on the quiet that they think these progressive—humph—methods are terrible, and just don’t educate and never will. But they can’t say so in public, because if they did they’d lose their jobs.”

In today’s context, the part of this that most jumps out at me is the mothers’ and editors’ confidence that these poor practices and results are not the teachers’ fault. Indeed, these methods are being imposed on teachers. It’s a sad tale that I continue to hear—teachers who have to close their doors and find spare moments to bring rigor and research-based practices to their classrooms.

Like E. D. Hirsch, I find today’s blame-the-teacher rhetoric shocking and disheartening. How did we get to this point? Hirsch offers a compelling explanation:

The favored structural reforms haven’t worked very well. The new emphasis on “teacher quality” implies that the reforms haven’t worked because the teachers (rather than the reform principles themselves) are ineffective. A more reasonable interpretation is that reforms haven’t worked because on average they have done little to develop “rich content knowledge within and across grades.”

If we are to improve the education we offer all children, reformers must stop blaming teachers and start working with them. As Hirsch explains, “The single most effective way to enhance teacher effectiveness is to create a more coherent multi-year curriculum, so that teachers at each level will know what students have already been taught.” A cumulative, rigorous curriculum is not a cure-all, but it is an essential platform for teachers to work together within and across grades. Schools can choose to write their own curriculum, adopt one, adapt a few—whatever works for them, so long as the result is a content-specific, coherent, cumulative body of knowledge and skills to be learned in each grade. Such a curriculum narrows the gaps in children’s abilities, makes differentiation more doable and effective, and enables the school community to deeply understand and support each child’s year-to-year progress.

In reform circles, however, curriculum is rarely discussed. Rather than wade into the hot water of precisely what students ought to learn, most reformers tinker around the edges of the educational enterprise (which boils down to what gets taught and what gets learned). To that, I say Humph! It’s the reformers’ ideas that are ineffective—not the hardworking teachers.

shutterstock_18732913

Stop blaming teachers for reformers’ faulty ideas.

(Photo courtesy of Shutterstock.) 

Testing: From the Mouths of Babes

by Lisa Hansel
May 8th, 2014

“No one learns from state tests. It’s testing what you know. You’re not learning anything from it.”

 —12th grader

“I like math or spelling tests better [than state accountability tests] because you can study for them. For the [state accountability tests], I wonder what will be on them this time.”

 —5th grader

“I like pre- and post-tests because you get to see the progress you’ve made.”

—4th grader

Is it just me, or do these kids know a whole lot more about assessment and increasing educational achievement than most state and national policymakers? Far too many policymakers seem to have lost sight of the most important goal of assessment and accountability: increasing learning. They seem stuck on accountability for the sake of accountability, unwilling to ask whether assessment dollars could be used more effectively.

I’m not against accountability—and I think assessment is necessary—but I am for allocating time and money in the most effective ways. So I find these students’ thoughts, and the new study in which they appear, pretty compelling. The study is Make Assessment Matter, by the Northwest Evaluation Association in cooperation with Grunwald Associates LLC. It explores students’ (4th – 12th graders), teachers’, and administrators’ views on all sorts of testing—from classroom quizzes to state accountability tests. Conclusion: “There is an urgency felt on the part of students, teachers and district administrators to emphasize assessment for learning rather than for accountability. The overwhelming preference for all parties is that assessment results be used to inform learning.” Sadly, today’s state tests not only don’t inform learning, they seem to be impeding it: “teachers (70 percent) and district administrators (55 percent) … [say] that the focus on state accountability tests takes too much time away from learning.”

Think about the weeks that are lost to state accountability tests each year as you absorb these key findings:

On the one hand, the vast majority of students, boys and girls, say they try hard on most tests and care about doing well on tests, among other findings that indicate how seriously they take tests and learning. On the other hand, some boys (46 percent) and girls (39 percent) say that tests are a waste of time.

It’s clear that students feel that certain kinds of tests are not very relevant to their learning, and so it’s not surprising to hear some students identify tests as a waste of time. In tandem with other findings, the message is clear: students want high-quality, engaging assessments that are tightly connected to learning….

Like students, teachers and district administrators would prefer to focus on tests that inform student learning. Most teachers (54 percent), and the vast majority of district administrators (89 percent), say that the ideal focus of assessments should be frequently tracking student performance and providing daily or weekly feedback in the classroom. This sentiment tracks with students’ attitudes about tests. Students express overwhelming agreement that tests are important for helping them and their teachers know if they are making progress in their learning and for understanding what they are learning.

Teachers say that teacher-developed classroom tests, performance tasks and formative assessment practice work best for supporting student learning in their classrooms, while state accountability tests are the least effective.

For an assessment to matter, it has to be directly tied to what is being studied in the classroom. For students to care about it, they need to be able to study for it and use the results in meaningful ways.

shutterstock_190861118

Image courtesy of Shutterstock.

That sounds perfectly reasonable to me. So, what are the logical implications for states? I see two options. One is to use Advanced Placement as a model: create detailed, content-specific courses and develop tests that only assess material in the course. I know it’s unheard of in state accountability testing, but I am actually being so crazy as to say that states should test students on the topics, books, people, ideas, events, etc. that they have been taught.

If state policymakers can’t stomach the idea of specifying the content to teach and test—if they can’t honor students’ desire to be tested in ways that inform learning—then they must honor students’ desire to not have their time wasted: make the tests zero stakes with zero test prep (like NAEP). Any test that is not tied to the specific content being studied in the classroom is a test of general knowledge and skills. Such a test can provide an informative snapshot of students’ and schools’ relative performance (and thus which schools and communities are in need of added supports). It can’t, however, indicate how any one student acquired her knowledge and skills (could be the teacher, the tutor that mom hired in October, the soccer coach who demands higher grades, the new librarian in town, finally being given eyeglasses, etc.). And therefore it can’t offer any precise indication of either teacher quality or how the student could improve. If a state wants to give a test that measures general abilities and provides nothing more than a snapshot and a trend line, that’s fine—provided the stakes and the prep time are minimized.

My preference, obviously, is for option one—especially if states would have the good sense to involve hundreds of educators in developing the specific content to be taught and assessed. Not only would the state-controlled, culminating test be useful for learning, in preparing for it teachers could use effective practices like frequent quizzing on essential content.

 

Federal Policy, Teacher Effects

by Lisa Hansel
March 5th, 2014

My objective today is to put words in a few prominent researchers’ mouths—or better yet, their paper, “Learning that Lasts: Unpacking Variation in Teachers’ Effects on Students’ Long-Term Knowledge.” Benjamin Master, Susanna Loeb, and James Wyckoff have posted online a “preliminary draft,” which no one is supposed to cite. Blogging, I assume with all online content, is fair game. This paper is terrifically important. I just want to see a draft that more fully discusses the many factors that contribute to teacher effects.

Let’s start with why this paper is worth your time: It’s a blockbuster for those worried about the negative consequences of annual high-stakes test-based teacher evaluations. Looking at the long-term impact of teachers with high value added, the researchers conclude:

Evaluation and accountability systems may incentive educators to focus excessively on short-term tested outcomes in ways that are not ultimately beneficial for students…. Collectively, this body of evidence demonstrates that teachers’ instructional practices can influence their short-term value-added performance in ways that do not correspond with long-term success for students…. Overall, our results demonstrate that teachers’ effects on students’ long-term skills can vary substantially and systematically, in ways that are not fully captured by short-term value-added measures of instructional quality.

We clearly need education policy to incentivize (or at least not impede) meaningful educational gains, so I hope policymakers will heed this research.

To increase the odds that they will heed it, the paper needs one quick little addition: more forceful acknowledgement that teachers’ effects are influenced by many factors. Several federal and state policies could be explored as means of positively influencing curriculum and instruction. This is not simply a teacher issue. It is a standards, curriculum, assessment, accountability, teacher preparation, professional development, leadership, and resource-allocation issue.

Is it really these researchers’ job to remind readers of the broader context? No. It’s just something that, given the importance of the issue, I’m hoping they’ll want to note.

Barack_Obama_through_a_magnifying_glass

When it comes to teacher effects, context matters.

Reading this paper, it’s easy to get swept up in thinking the teacher makes all the difference. For example, the more academic knowledge teachers have, the more they seem to infuse that in their instruction, to great effect:

The within-subject value-added persistence of ELA teachers who attended a more competitive undergraduate institution is significantly and substantially higher than that of teachers who attended a less competitive institution…. Differences in persistence are similarly large when comparing teachers whose SAT Verbal exam scores or LAST licensure exam scores are in the top third of the teacher distribution, in comparison to lower-scoring teachers. In both cases, higher scoring teachers show greater persistence…. It is notable that our teacher ability characteristics predict large differences in ELA teachers’ value-added persistence, even though they are not themselves correlated with teachers’ short-term value-added effects.

One might be tempted to think these direct teacher effects are simply teacher-quality issues. But nothing in education is so simple. Ask yourself: what’s likely to make a person with the potential to have lasting effects want to be a teacher for years to come? Rigorous standards and engaging curriculum, meaningful assessments that support instruction, accountability policies that don’t incentivize test prep, academically demanding preparation programs, tailored professional development, helpful leaders, etc.

Much of the paper is devoted to examining potential student- vs. teacher-level drivers of the variation in teachers’ long-term impact. That, obviously, is a key question—I just want to see more acknowledgement that the “teacher effects” are federal-, state-, district-, and school-policy effects. Here’s the heart of the research:

Observable student characteristics related to their socio-economic status or prior ability also predict substantial variation in their ELA teachers’ value-added persistence. The persistence of achievement gains coming from having an effective teacher is far lower for students who are eligible for free lunch, are black or Hispanic, or whose twice-lagged ELA achievement scores are below the mean…. These students may be receiving ELA instruction that is less focused on long-term knowledge, or they may be less skilled at acquiring or retaining long-term knowledge.

Ultimately, the researchers conclude the primary issue is likely “instruction that is less focused on long-term knowledge”:

We see evidence of the importance of instruction in the positive association between teachers’ academic ability and their contributions to students’ long-term knowledge. Even more compelling, we find that schools that serve more disadvantaged students or that hire fewer of these high-ability teachers have lower value-added persistence in ELA for all of their students. Students, regardless of their prior test performance, who attend schools with many low-performing students demonstrate lower persistence of the learning gains they achieve from having a high value-added teacher. The persistence in low-achieving schools is less than half the rate of that in other schools. These findings provide evidence that instructional quality is a key driver of the variation that we observe in value-added persistence, and that school-level curriculum or instructional norms may foster differences in instructional quality. Unfortunately, we are unable to directly observe the instructional practices of teachers or schools in our sample. However, in light of prior research on educators’ responses to high stakes accountability pressures … one plausible explanation for our findings could be that schools serving lower performing students systematically prioritize gains in short-term tested achievement in ways that detract from teachers’ focus on long-term knowledge generation.

As I’ve said, there’s a whole lot beyond “school-level curriculum or instructional norms” that “may foster differences in instructional quality.” The authors of this paper know that—and it’s certainly not their fault that many policymakers need to be reminded. But they do. And if more policymakers get the message that we have a multifaceted, highly complex problem to address, perhaps more desperately needed research dollars will be provided and more varied policies will be piloted.

Policymakers: Stop Being Agnostic about Curriculum

by Lisa Hansel
January 29th, 2014

This post originally appeared on Common Core Watch, a blog by the Thomas B. Fordham Institute.

 

Pop quiz! Which of the following statements is in the Common Core State Standards?

(a) Through extensive reading of stories, dramas, poems, and myths from diverse cultures and different time periods, students gain literary and cultural knowledge.

(b) By reading texts in history/social studies, science, and other disciplines, students build a foundation of knowledge in these fields that will also give them the background to be better readers in all content areas.

(c) At a curricular or instructional level, texts—within and across grade levels—need to be selected around topics or themes that systematically develop the knowledge base of students.

(d) Having students listen to informational read-alouds in the early grades helps lay the necessary foundation for students’ reading and understanding of increasingly complex texts on their own in subsequent grades.

(e) All of the above.

The answer is e, all of the above. Knowledge is the key to reading comprehension. It’s the key to college, career, and citizenship readiness. It’s the key to meeting the Common Core standards. (see pages 10 and 33of the standards—and for even more on building knowledge, see page 6 and Apendix A page 33).

To be even more blunt, the standards require a “content-rich curriculum” (page 6) that is “intentionally and coherently structured to develop rich content knowledge within and across grades” (page 10).

If you are a master teacher with a supportive administrator and collaborative colleagues, the standards give you all the guidance you need. Between the model on page 33 and the research summary in Appendix A, there’s a clear vision for creating a curriculum that systematically builds knowledge.

Knowledge-driven careers courtesy of Shutterstock.

But if you are a state-level policymaker or district superintendent, the path forward is murkier. You don’t want to mandate a curriculum, but you do need to encourage all schools to adopt, adapt, or create more rigorous, coherent, knowledge-building curricula. What to do? Four models are worth considering—two at the state level and two at the district level.

Focus on Alignment: Massachusetts

Starting in the 1990s, Massachusetts began taking the whole idea of a standards-based education system very seriously. The Bay State created instructional frameworks that were (relative to other states, if not to many other nations) very content specific. What students had to learn was clear, which enabled teachers to collaborate on a much deeper level. Policymakers got three big things right: First, they did not mandate pedagogy. Second, they actually based the MCAS exams on the instructional frameworks. (Many other states had standards and assessments, but the standards were so vague that virtually any assessments could claim to be aligned with them. As a result, the standards did not truly guide instruction, setting up an assessment-based guessing game for teachers.) Third, they stayed the course for many years—standing firm against allegations that the standards were too high and the tests too hard and, crucially, being far more supportive than punitive. For many years, the emphasis was on framework-based teacher preparation and ongoing professional development. The results (nationally and internationally) have been spectacular.

Provide a Model: New York

While I can empathize with educators who feel that New York is moving too fast with the Common Core, I must also credit the state for heading in the right direction. New York realized that the standards would mean major instructional shifts, and has been working to provide—but not mandate—curricular resources to help teachers make those shifts. The EngageNY website is a rich resource; teachers throughout the Empire State and far beyond are using it to better understand the Common Core. (Full disclosure: Core Knowledge Language Arts was chosen by New York as the model ELA curriculum for preschool–second grade implementation of the standards.) Massachusetts took about a decade to fully implement its standards-based system; I predict that New York will figure out ways to heed educators’ concerns while staying the course.

Build Your Own: Washington, D.C.

Like New York, the District of Columbia realized that the Common Core requires a content-rich curriculum. It also saw many benefits for students and teachers when a district has a shared, specific, curricular plan: Students endure fewer gaps and repetitions when they change schools and teachers are able to learn more from each other. Being large enough to have adequate resources and small enough to engage in district-wide initiatives, the District of Columbia Public Schools has gotten teachers involved in writing Common Core–aligned Scope and Sequence guides for each grade. This is especially important because of the city’s high rates of teacher turnover and student mobility. States that don’t want to follow New York’s path could incentivize districts to follow D.C.’s path. Even a small initiative, such as funding three to five districts, would help the whole state by creating multiple curricular models for other districts to adopt or adapt.

Invest in R&D: New York City

A few years before the Common Core, New York City tiptoed into analyzing the efficacy of different curricula. This is worth mentioning not because of the quality of the study (a small pilot) and not because of the programs being tested (Core Knowledge Language Arts was one), but because comparisons of curricula are desperately needed. As Brookings scholars Russ Whitehurst and Matt Chingos have explained, instructional materials can have as large an impact on learning as teacher quality—and programs are much easier to change than people—yet little is known about various materials’ relative effectiveness. Large districts like NYC—and all districts with state support—have the capacity to conduct more and better research. To make the most of the Common Core, we need to create content-rich curricula and commit to an ongoing R&D process that drives continuous improvement in curriculum and instruction.

Curriculum alone, no matter how good, is no silver bullet. But it should be the foundation for all other education work. From teacher preparation and professional development to assessment and accountability to student remediation and enrichment, the education enterprise is more effective and efficient when it rests on a clear statement of what students are to learn in each grade.

 

What Really Matters Most?

by Lisa Hansel
January 23rd, 2014

This post originally appeared on Peter Meyer’s education policy blog IdeaLab, hosted by the CUNY Institute for Education Policy at Roosevelt House.

 

When asked what matters most to me, I quickly answer: my family and friends. That’s appropriate, but if I were being accurate, I’d have to start with oxygen. That’s not what anyone wants to hear—but it is true.

I see a parallel situation in discussions of school improvement. In casual discussions and even serious debates, there seems to be a de facto, appropriate answer as to what matters most in creating a good school: great teachers and supportive parents. Now, I’m not going to say these things are unimportant; just like my family and friends, they are essential. But is there a more accurate answer, one that, like oxygen, is taken for granted? I think there is: the content of the curriculum, the specific knowledge and skills taught each day.

My hunch is that curriculum is glossed over in different ways by educators and policy wonks.

For educators, the content of the curriculum really is like oxygen. Teaching is always about something, and that something has to be specified before any other decisions can be made. That’s so obvious that it’s assumed, prompting educators to jump to other factors in thinking about what’s essential to a great school. Now, don’t get me wrong: the curriculum doesn’t make a school great all by itself any more than oxygen alone makes me live. Both are merely the necessary preconditions. Yet while it is possible to find a bad school with a great curriculum, it is no more possible to find a good school with a bad curriculum than a human being who can live without oxygen. When educators take the content of the curriculum for granted, they lose opportunities to coordinate and collaborate. Students may be learning something valuable in each grade or course, but they do not receive the benefits of a coherent, cumulative, cross-curricular experience.

Many policy wonks, on the other hand, seem to have no idea that curriculum matters. Some don’t even realize that standards and curricula are not the same thing. Theoretically, I could blame the educators for not explaining to the policymakers that curriculum is like oxygen—but in the real world I can’t. In the 100%-proficient-or-else era, what sane educator would encourage policymakers to mess with their oxygen? Unfortunately, omitting questions about the curriculum virtually ensures that the standards regime cannot attain its goal of raising student proficiency. Why is this?

It’s been almost five years since Russ Whitehurst wrote “Don’t Forget Curriculum,” noting that “policy makers who cut their teeth on policy reforms in the areas of school governance and management rather than classroom practice, [are] people who may be oblivious to curriculum for the same reason that Bedouin don’t think much about water skiing.” Importantly, Whitehurst compared the impact of curricular improvements to that of other reforms, such as charter schools, altering the teacher workforce, preschool, and state standards. Conclusion: “Curriculum effects are large compared to most popular policy levers.”

This is why I am still trying to mess with the oxygen: it is the necessary precondition for improving schools, closing the achievement gap, engaging parents, and preparing teachers.

Trying again a couple of years ago, Whitehurst and Matt Chingos published “Choosing Blindly: Instructional Materials, Teacher Effectiveness, and the Common Core.” This time, there was a cool graphic tightly focused on curriculum vs. teacher quality, the clear leader in appropriate-but-inaccurate discussions of what matters most:

Since curriculum matters, let’s start acting like it matters:

  • Researchers: do more longitudinal, well-designed studies that compare curricula.
  • Policy wonks: don’t mandate a curriculum, but support efforts—from the school level to the research university level—to constantly improve curricula.
  • Assessment developers: stop pretending like assessments are curriculum neutral. Each test question contains specific content and favors students who happened to be taught that content. So long as assessments are intentionally designed to have the content of the questions be unpredictable, the only way to prepare for them is to systematically and efficiently build broad knowledge.
  • Teacher-quality hawks: realize that sometimes good people are forced to use bad programs and practices. The surest path to better teaching is better curriculum. If a curriculum with strong evidence of effectiveness is not working in a particular classroom, that’s cause for investigation (but not for jumping to conclusions).
  • Educators: within schools, work together to adopt, adapt, or create a coherent, grade-by-grade curriculum that maximizes cross-discipline connections and efficiently builds knowledge and skills. Across schools in areas with high student mobility, agree to a set of specific knowledge and skills to be taught in each grade; children who change schools will benefit immediately—and so will their teachers.
  • Parents: get a copy of your school’s curriculum and ask how you can supplement it at home.
  • Librarians: get copies of the curricula of the schools in your area and pull together supportive and supplemental resources.
  • Everyone: stop taking our oxygen for granted.

Everyone can and should be an oxygen hawk.

Miracle on High Street

by Lisa Hansel
December 19th, 2013

2013 has been a miraculous year for the folks at 801 E. High Street, the beautiful old house that the Core Knowledge Foundation calls home.

Core Knowledge Language Arts went from a pilot program to a major model for Common Core implementation. E. D. Hirsch’s ideas—and, more importantly, the research supporting them—earned a new audience. Plus, the need to close the vocabulary gap gained a prominent champion (as well as several close allies).

Just when we thought it couldn’t get any better, last week was phenomenal. I can’t rank order the events, so I’ll go in chronological order.

First, the Albert Shanker Institute hosted a forum on “The Word Gap & the Common Core” in which Susan Neuman hammered home the need to systematically build knowledge and vocabulary in early childhood. As I wrote over at the Shanker Blog, Neuman kicked off with the perfect metaphor: Words are just the tip of the iceberg. The concepts and knowledge—and the opportunities to acquire them—are underneath the words. In just 15 minutes (which you can watch online), Neuman explained that the vocabulary gap is actually a knowledge gap and set forth a clear path for closing it. Spoiler alert: the research only provides one way to do it—grouping challenging texts by topic and immersing young children in those texts though read-alouds and meaningful conversations. Sounds familiar.

Second, Michael Petrilli went for the hard sell on Core Knowledge in the New York Daily News. Writing to NYC’s mayor-elect Bill de Blasio, Petrilli was blissfully blunt:

As scholar and Core Knowledge creator E.D. Hirsch, Jr. has argued for 30 years — and as more recent cognitive science has confirmed — knowledge is the building block of literacy. Once students learn to “decode” the English language, their ability to comprehend what they read is all about what they know….

The job of elementary schools, then, should be to systematically build students’ content knowledge in important areas like history, geography, civics, science, art, music and literature. Yet most elementary schools (nationwide — not just in New York) are content-free wastelands….

Bloomberg’s Department of Education has listed Core Knowledge as one of the model curricula for New York City teachers to consider as they transition to the new standards.

De Blasio should go even further. If he wants to be bold, he might urge all city elementary schools to adopt Core Knowledge.

Third—proof that good things come in threes—Joel Klein said that “The best parts of the Common Core are tethered to Core Knowledge.” (See for yourself!) Speaking with David Steiner in a forum at the CUNY Institute for Education Policy, Klein was candid about his tenure as NYC schools chancellor. He regrets that at the beginning, he did not know how important it is to build knowledge and vocabulary in the early grades. But after doing some reading, especially Sol Stern’s powerful critiques, Klein said he reached out to E. D. Hirsch. Klein’s pilot study of Core Knowledge Language Arts (CKLA) was soon underway, with the CKLA schools consistently outperforming the comparison schools. CKLA is now being recognized as a model for the type of curriculum called for by the Common Core standards: one that “is intentionally and coherently structured to develop rich content knowledge within and across grades.”

Klein expects that to have a “national impact.” We’ll keep working to make sure his words ring true in 2014.

 

Can Knowledge Level The Learning Field For Children?

by Guest Blogger
December 2nd, 2013

By Esther Quintero

Esther Quintero is a senior research fellow at the Albert Shanker Institute. This post first appeared on the Shanker Blog.

How much do preschoolers from disadvantaged and more affluent backgrounds know about the world and why does that matter? One recent study by Tanya Kaefer (Lakehead University) Susan B. Neuman (New York University) and Ashley M. Pinkham (University of Michigan) provides some answers.

The researchers randomly selected children from preschool classrooms in two sites, one serving kids from disadvantaged backgrounds, the other serving middle-class kids. They then set about to answer three questions:

  1. Do poor and middle-class children possess different knowledge about the world?
  2. Do differences in knowledge influence the children’s ability to learn in the classroom?
  3. If differences in preexisting knowledge were neutralized, would the two groups of children learn similarly?

To answer the first question, the researchers determined how much children from both groups knew about birds and the extent to which they were able to make inferences about new words based on such knowledge.

Not surprisingly, lower-income children had significantly less knowledge about birds and bird behaviors than did their middle-class peers. To rule out the possibility that these differences were the result of disparities in language proficiency, Kaefer et al. measured the children’s receptive vocabularies. This way, they were able to establish that poor kids knew less about birds, not merely because they knew fewer words related to birds, but because they had less information about the domain in general.

To answer the second question — whether differences in knowledge influence the kids’ ability to learn in the classroom — a second study evaluated children’s ability to understand words out of context and to comprehend a story that was read to them. As predicted, children from middle-class backgrounds, who had greater knowledge about the domain category (i.e., birds), performed better in these two tasks than children with more limited knowledge about the domain.

It may not be obvious to adults, but learning words from books is not an automatic or straightforward task for young children. In fact, argue the authors of the paper, one of the factors influencing this process is children’s preexisting knowledge. Previous research (cited in the paper) has established that children with larger vocabularies acquire new words implicitly from storybooks more readily than children with smaller vocabularies. At least two mechanisms might explain the relationship between vocabulary and learning.

First, the authors note, one possible explanation is that metalinguistic factors (e.g., verbal IQ, working memory) explain the relationship between vocabulary knowledge and implicit word learning.

Alternatively, if children’s vocabulary is viewed as an indicator (or “reflection”) of their general background knowledge, it may be the breadth and depth of their preexisting knowledge that influences their implicit word learning.

The logic of the second mechanism is as follows: Children’s preexisting knowledge creates a framework that facilitates the acquisition of new information; knowing more words and concepts scaffolds children’s ability to slot new information in the “right places,” and to learn related words and concepts more efficiently.

To recap, the first study discussed above established that children from disadvantaged backgrounds know less about a topic (i.e., birds) than their middle-class peers. Next, in study two, the researchers showed that differences in domain knowledge influenced children’s ability to understand words out of context, and to comprehend a story. Moreover, poor kids—who also had more limited knowledge—perform worse on these tasks than did their middle-class peers. But could additional knowledge be used to level the playing field for children from less affluent backgrounds?

In study three, the researchers held the children’s prior knowledge constant by introducing a fictitious topic—i.e., a topic that was sure to be unknown to both groups. When the two groups of children were assessed on word learning and comprehension related to this new domain, the researchers found no significant differences in how poor and middle-class children learned words, comprehended a story or made inferences.

These results:

  • Add to the body of research showing that preexisting knowledge shapes incidental vocabulary learning and comprehension for children, and that this is true for children as young as preschool age;
  • Highlight the need to build children’s background knowledge more systematically and strategically, and suggest that procedures to activate children’s prior knowledge—e.g., storybook reading—may prove fruitless when such knowledge does not exist.

While this research, like all research, has limitations—see the paper for a discussion of these—the results taken together suggest that one powerful way to level the “learning field” for all children is to facilitate poor kids’ access to “taken for granted” knowledge that middle class children, on average, are more likely to possess, primarily because they have been exposed to it in the first place.

When poor and middle-class children are given the same opportunities to assimilate new knowledge, their subsequent learning is comparable. Of course this is only one study, but the main finding and its implications are extremely powerful. It suggests that if preschool programs are not making a difference for children from disadvantaged backgrounds, it might be the case that the programs are not tackling an important but solvable problem: A deficit in knowledge.

 

A Moving Problem

by Lisa Hansel
October 21st, 2013

Terrifying. Half-way through first grade, I was plucked out of a Montessori program that was part of the Montgomery County, MD, school district and dropped into Page Jackson Elementary School in Jefferson County, WV. Everyone was perfectly nice, helpful even—but I was a shy kid in a strange land.

Jarring is too mellow a word. Having gone to that Montessori program for kindergarten as well, I only had one concept of what school was. Page Jackson did not fit that concept. It was bigger and more traditional; finding my way from the front door to my classroom took me weeks to master. Too fearful to speak up for myself, for several days my inability to print was mistaken for an inability to write. No one ever asked if I wrote in cursive, if I had taught younger children some phonics and writing strokes, if I had memorized the times tables up to 10×10.

Lucky me—my mother was there to speak up for me. Soon enough, the ways in which I was ahead were appreciated and the ways in which I was behind were remediated.

Most children who change schools are not so lucky. The research is clear: changing schools is strongly associated with lower performance. The more kids move, the less they learn.

(Image courtesy of Shutterstock.)

Now a new report, The Invisible Achievement Gap, shows that changing schools is devastatingly frequent for some of our most at-risk students: youth in foster care. Here’s a quick summary:

Only about two thirds of students in foster care attended the same school for the full school year. In contrast, over 90 percent of the low-SES [socioeconomic status] and the statewide student populations attended the same school all year. Furthermore, about 1 in 10 students in foster care attended three or more schools during the school year, a level of school mobility experienced by only about 1 percent of the low-SES and general student populations….

Students in foster care, like low-SES students, were consistently more likely than the general population to attend the state’s lowest-performing schools and less likely to attend the state’s highest-performing schools….

CST [California Standards Test] results showed that students in foster care consistently fell far short of achieving proficiency in English language arts, elementary mathematics, and the secondary mathematics courses algebra I and algebra II…. They were consistently outperformed by low-SES students. Test results for students in foster care fell into the two lowest performance levels for English language arts and mathematics—below basic and far below basic—at twice the rate of those for the statewide student population….

Students in foster care were more likely than all comparison groups to drop out.

These results are terrible! Yet these data also point to one major way to help:

The majority of California students in foster care were enrolled in just a small number of districts. Specifically, two thirds of these students were enrolled in 10 percent of the state’s school districts, with each of these districts enrolling at least 100 students in foster care.

I know it’s heresy, but here goes: All of the schools in that 10 percent of districts should teach the same core curriculum. School districts can’t prevent the need for foster care or the school transfers that result. They can make those transfers much easier on their students. If all of the schools in these high-foster-care districts shared a core curriculum, then students could change schools without having to change what they are learning. Any days missed would be easier to address, and teachers would have common ground for helping each other serve these neediest of students.

As the report states, foster care students are already likely to “attend the state’s lowest-performing schools.” This shared curriculum project could be the foundation for a renaissance for these schools. Not only would mobile students have fewer gaps in their knowledge, the districts could collaborate on instructional materials and professional development.

Districts that share students already have shared responsibilities. Why not share curriculum and wisdom too?