Curriculum Doesn’t Matter, Unless You Care about Achievement and Mobility

by Lisa Hansel
September 11th, 2014

Five years ago, Russ Whitehurst published an important paper comparing the effects of various education reforms. Better teachers and curriculum rose to the top, with what is taught being just as important as who is doing the teaching. But that finding didn’t fit with reformers’ obsession with teachers, so the paper was largely ignored. Two years ago, Whitehurst and Matthew Chingos did a more extensive look, confirming the previous findings and challenging states to begin gathering data on which materials are being used in schools.

Now, with pressure to interpret and meet the Common Core standards, curriculum, textbooks, and other instructional materials are finally getting wider attention. Much of that attention has been negative, as those who are against the standards seem to enjoy finding misinterpretations of the standards’ intent. I find that gotcha game silly; no one really expects initial stabs at Common Core–aligned materials to be terrific. Over time they will improve—and with support they will improve more quickly.

I’m thrilled to see growing interest in providing that support. The Helmsley Charitable Trust, the Hewlett Foundation, and the Bill & Melinda Gates Foundation are leading the way by funding several efforts, the most promising of which just might be EdReports.org. Preparing to launch in the winter, EdReports.org is involving teachers in intensive reviews of K–12 math and ELA materials, and the reviews will be free online. At the same time, Rachel Leifer and Denis Udall, program officers with Helmsley and Hewlett (respectively), have penned a plea for “Disrupting the Textbook Status Quo.” Since Core Knowledge is a small non-profit trying to offer better materials for free online, I am heartened by their call for more philanthropies to support development and dissemination efforts so that “a marketplace for instructional materials that rewards quality and innovation” can be created.

For philanthropies that aren’t quite convinced that curriculum matters, here’s one more study to add to the great work noted by Leifer and Udall. In “The Role of the School Curriculum in Social Mobility,” Cristina Iannelli shows that the content of the curriculum has lasting effects.* This study is important not only because of its findings, but because relatively few studies look at the actual courses students take. Iannelli is a professor in the UK; she used the UK’s National Child Development Study (NCDS), which tracks all babies born in the UK in 1958, gathering data at ages 7, 11, 16, 23, 33, 42, 46, and 50. As Iannelli writes (p. 910):

The people in the study were in secondary schools between 1969 and 1976 during the period of reorganisation of British secondary education from a selective to a comprehensive system. The coexistence of different secondary school systems at the end of the 1960s provides an excellent opportunity to study the effect of studying different curricula and attending different types of school on individuals’ chances of reaching the highest social classes of destination.

Focusing on social and occupational status at ages 23, 33, and 42, Iannelli’s findings at age 23 were what you’d expect: parental education and school selectivity had a big impact. Curriculum did too, but what’s really interesting is that the relative importance of curriculum went up—and the importance of parental education and school selectivity went down—as people aged (p. 923–924):

Selective schools, languages, English, mathematics and science subjects had a positive and significant effect on the chances of being in the top social classes and reduced the chances of entering the bottom classes…. [The] results suggest that the indirect effects of parental education via school types and curricula are stronger at the beginning of respondents’ occupational career than at later stages. The opposite is true for social class of origin: it is in the long run that school and curricular choices emerge more powerfully as transmitters of social advantages….

We tested whether the effect of curriculum and school type at age 33 and 42 was simply a result of their effect at age 23 and 33…. The results, before and after controlling for prior occupational destinations, barely change in the analysis of class of destination at 33, indicating that the effect of studying different subjects and attending various types of schools continues beyond the point of career entry. However, when analysing destinations at age 42 after controlling for destination at age 33, while the effect of subjects remains the same the effect of school types reduces and is no longer significant. These results suggest that the subjects studied at school are very good predictors of individuals’ destinations at all three stages of occupational career. On the other hand, the school type attended has a significant short-term and medium-term effect on individuals’ occupational destinations but they become less important for explaining later destinations. This may indicate that cognitive effects may be more persistent than institutional status effects…. The long-lasting effects of some school subjects may indicate that they provide skills, such as critical thinking and complex reasoning, which are useful for individuals’ future occupational careers. [Emphasis added.]

Studying languages, English, mathematics, and science does indeed enhance critical thinking and complex reasoning abilities. In fact, these crucial abilities can only be increased by developing rich knowledge. Cognitive science on how knowledge builds on knowledge, and skills depend on knowledge, would predict these occupational findings. The students who took many courses in these subjects began their careers with a solid foundation of knowledge and skills that enabled them to learn and grow at work.

We won’t ever be able to predict where each young person’s career will go, but we have plenty of evidence as to the type of education that offers the best preparation: a broad, rich, academic curriculum that builds content knowledge and skills together.

shutterstock_196046843

Mobility by knowledge courtesy of Shutterstock.

 

* Many thanks to Webs of Substance for finding this study!

What Really Matters Most?

by Lisa Hansel
January 23rd, 2014

This post originally appeared on Peter Meyer’s education policy blog IdeaLab, hosted by the CUNY Institute for Education Policy at Roosevelt House.

 

When asked what matters most to me, I quickly answer: my family and friends. That’s appropriate, but if I were being accurate, I’d have to start with oxygen. That’s not what anyone wants to hear—but it is true.

I see a parallel situation in discussions of school improvement. In casual discussions and even serious debates, there seems to be a de facto, appropriate answer as to what matters most in creating a good school: great teachers and supportive parents. Now, I’m not going to say these things are unimportant; just like my family and friends, they are essential. But is there a more accurate answer, one that, like oxygen, is taken for granted? I think there is: the content of the curriculum, the specific knowledge and skills taught each day.

My hunch is that curriculum is glossed over in different ways by educators and policy wonks.

For educators, the content of the curriculum really is like oxygen. Teaching is always about something, and that something has to be specified before any other decisions can be made. That’s so obvious that it’s assumed, prompting educators to jump to other factors in thinking about what’s essential to a great school. Now, don’t get me wrong: the curriculum doesn’t make a school great all by itself any more than oxygen alone makes me live. Both are merely the necessary preconditions. Yet while it is possible to find a bad school with a great curriculum, it is no more possible to find a good school with a bad curriculum than a human being who can live without oxygen. When educators take the content of the curriculum for granted, they lose opportunities to coordinate and collaborate. Students may be learning something valuable in each grade or course, but they do not receive the benefits of a coherent, cumulative, cross-curricular experience.

Many policy wonks, on the other hand, seem to have no idea that curriculum matters. Some don’t even realize that standards and curricula are not the same thing. Theoretically, I could blame the educators for not explaining to the policymakers that curriculum is like oxygen—but in the real world I can’t. In the 100%-proficient-or-else era, what sane educator would encourage policymakers to mess with their oxygen? Unfortunately, omitting questions about the curriculum virtually ensures that the standards regime cannot attain its goal of raising student proficiency. Why is this?

It’s been almost five years since Russ Whitehurst wrote “Don’t Forget Curriculum,” noting that “policy makers who cut their teeth on policy reforms in the areas of school governance and management rather than classroom practice, [are] people who may be oblivious to curriculum for the same reason that Bedouin don’t think much about water skiing.” Importantly, Whitehurst compared the impact of curricular improvements to that of other reforms, such as charter schools, altering the teacher workforce, preschool, and state standards. Conclusion: “Curriculum effects are large compared to most popular policy levers.”

This is why I am still trying to mess with the oxygen: it is the necessary precondition for improving schools, closing the achievement gap, engaging parents, and preparing teachers.

Trying again a couple of years ago, Whitehurst and Matt Chingos published “Choosing Blindly: Instructional Materials, Teacher Effectiveness, and the Common Core.” This time, there was a cool graphic tightly focused on curriculum vs. teacher quality, the clear leader in appropriate-but-inaccurate discussions of what matters most:

Since curriculum matters, let’s start acting like it matters:

  • Researchers: do more longitudinal, well-designed studies that compare curricula.
  • Policy wonks: don’t mandate a curriculum, but support efforts—from the school level to the research university level—to constantly improve curricula.
  • Assessment developers: stop pretending like assessments are curriculum neutral. Each test question contains specific content and favors students who happened to be taught that content. So long as assessments are intentionally designed to have the content of the questions be unpredictable, the only way to prepare for them is to systematically and efficiently build broad knowledge.
  • Teacher-quality hawks: realize that sometimes good people are forced to use bad programs and practices. The surest path to better teaching is better curriculum. If a curriculum with strong evidence of effectiveness is not working in a particular classroom, that’s cause for investigation (but not for jumping to conclusions).
  • Educators: within schools, work together to adopt, adapt, or create a coherent, grade-by-grade curriculum that maximizes cross-discipline connections and efficiently builds knowledge and skills. Across schools in areas with high student mobility, agree to a set of specific knowledge and skills to be taught in each grade; children who change schools will benefit immediately—and so will their teachers.
  • Parents: get a copy of your school’s curriculum and ask how you can supplement it at home.
  • Librarians: get copies of the curricula of the schools in your area and pull together supportive and supplemental resources.
  • Everyone: stop taking our oxygen for granted.

Everyone can and should be an oxygen hawk.

Choosing Curriculum Without Evidence

by Robert Pondiscio
April 13th, 2012

If you wanted to improve medical care, would you focus on hospital administration and patient insurance?  Or would you look at the treatment doctors were giving patients?  Would you try to improve a sports team’s won-loss record by focusing on stadium layout and the team’s travel schedule?  Then why, ask Brookings’ Matthew Chingos and Russ Whitehurst, do education policy makers focus most of their attention on academic standards, teacher evaluation, and school accountability policies?  Shouldn’t we be looking instead at instructional materials?

“There is strong evidence that the choice of instructional materials has large effects on student learning—effects that rival in size those that are associated with differences in teacher effectiveness,” the two write in a new paper from Brookings, Choosing Blindly: Instructional Materials, Teacher Effectiveness, and the Common Core.

“Whereas improving teacher quality through changes in the preparation and professional development of teachers and the human resources policies surrounding their employment is challenging, expensive, and time-consuming, making better choices among available instructional materials should be relatively easy, inexpensive, and quick.”

There’s one big hurdle to clear in correcting this rather obvious problem: Little effort has been made by the field to differentiate effective curricular materials from ineffective ones.  In fact, in most states, districts and schools, it’s nearly impossible to know what materials are being used at all.

“In every state except one, it is impossible to find out what materials districts are currently using without contacting the districts one at a time to ask them. And the districts may not even know what materials they use if adoption decisions are made by individual schools. The National Center for Education Statistics (NCES), which has the mission of collecting and disseminating information related to education in the U.S., collects no information on the usage of particular instructional materials.”

Chingos and Whitehurst predict the blindness on curriculum will become a critical problem for Common Core Standards implementation.  “Publishers of instructional materials are lining up to declare the alignment of their materials with the Common Core standards using the most superficial of definitions,” they note.  “The Common Core standards will only have a chance of raising student achievement if they are implemented with high-quality materials, but there is currently no basis to measure the quality of materials. Efforts to improve teacher effectiveness will also fall short if they focus solely on the selection and retention of teachers and ignore the instructional tools that teachers are given to practice their craft.”

The paper offers up a number of suggestions:  State education agencies should collect data from districts on the instructional materials in use in their schools.   also wants to see the National Governors Association (NGA) and Council of Chief State School Officers (CCSSO) “put their considerable weight behind the effort to improve the collection of information on instructional materials in order to create an environment in which states, districts, and schools will be able to choose the materials most likely to help students master the content laid out in the Common Core standards.”

Chingos and Whitehurst are dead-on in their critique of ed reform’s indifference to curriculum and materials.  When we focus on the mechanism by which schools are  created, managed, financed or evaluated, we are assuming that what kids learn, and with which materials, is pretty much settled, or doesn’t really matter.  All that’s left to do is figure out what works in terms of delivery of instruction and grow it, or figure out what doesn’t work and shut it down.  Any teacher who has worked with different literacy or math programs can easily attest this is not the case.

Whitehurst: Fund Curriculum Research

by Robert Pondiscio
June 11th, 2010

A new report by Brookings’ Russ Whitehurst offers four ideas which “offer substantial promise for improving American education, are achievable and have low costs.”  The first one is to ”choose K–12 curriculum based on evidence of effectiveness.” 

Little attention has been paid to choice of curriculum as a driver of student achievement. Yet the evidence for large curriculum effects is persuasive. Consider a recent study of first-grade math curricula, reported by the National Center for Education Evaluation and Regional Assistance in February 2009. The researchers randomly matched schools with one of four widely used curricula. Two curricula were clear winners, generating three months’ more learning over a nine-month school year than the other two. This is a big effect on achievement, and it is essentially free because the more effective curricula cost no more than the others.

There are myriad moving parts in a good educational outcome, especially for low-SES and minority children.  Curriculum is the least appreciated moving part and isn’t nearly as fun to fight about as structural reforms like charter schools, merit pay, tenure and accountabilty. Among the ed reformers who drive the policy agenda (and have never written a lesson plan), there is a strong tendency to see curriculum, as the National Review’s Jim Manzi put it, as “motherhood and apple pie” or simply see it subsumed within the teacher quality debate, i.e., good teachers make good curricular decisions.  Less appreciated is how a sequenced, structured curriculum can improve teacher quality by allowing teachers to hone their craft, focusing on delivering instruction and differentiation, rather than the enormously difficult and time consuming work of deciding what to teach, and aligning curricular decisions with state and local standards. Curriculum, as Whitehurst has consistently argued, is critical.

“The federal government should fund many more comparative effectiveness trials of curricula, and schools using federal funds to support the education of disadvantaged students should be required to use evidence of effectiveness in the choice of curriculum materials. The Obama administration supports comparative effectiveness research in health care,” Whitehurst notes.  “It is no less important in education.”

Hear, hear.   Whitehurst’s paper also calls for evaluating teachers “in ways that meaningfully differentiate levels of performance”;  accrediting online education providers so they can compete with traditional schools across district and state lines; and providing the public with information that will allow comparison of the labor-market outcomes and price of individual post-secondary degree and certificate programs.

Not Either/Or…It’s AND

by Robert Pondiscio
October 28th, 2009

At Eduwonk, Andy Rotherham catches up to Russ Whitehurst’s paper, Don’t Forget Curriculum.  But he misses the boat when he writes, “I’m not sure when curriculum and reforms like choice, teacher quality, etc…became either/or.”   I’m not sure where Andy’s getting that message, but it’s not from Russ Whitehurst, who went out of his way NOT to say that.  Here’s the relevant quote from his paper:

This is not to say that curriculum reforms should be pursued instead of efforts to create more choice and competition through charters, or to reconstitute the teacher workforce towards higher levels of effectiveness, or to establish high quality, intensive, and targeted preschool programs, all of which have evidence of effectiveness. It is to say that leaving curriculum reform off the table or giving it a very small place makes no sense.

Over at the American Enterprise Institute’s blog, Charles Murray adds his voice to the curriculum choir.

Curriculum: More Reform for Less Money

by Robert Pondiscio
October 15th, 2009

From Day One, among this blog’s raisons d’être has been to say to ed reformers of  every stripe “don’t forget curriculum.”  So it’s great to hear Brookings’ Russ Whitehurst say the same thing–and with cold, hard data to back it up.   In his latest Letter on Education, Whitehurst lays out an argument that should catch the eye of everyone who is focused on charter schools, teacher quality, early childhood ed and standards as the means of boosting student achievement.  He looks at the effect sizes of those reforms and reports curriculum effects have a much greater impact than all of them:

Further, in many cases they are a free good. That is, there are minimal differences between the costs of purchase and implementation of more vs. less effective curricula. In contrast, the other policy levers reviewed here range from very to extremely expensive and often carry with them significant political challenges, e.g., union opposition to merit pay for teachers. This is not to say that curriculum reforms should be pursued instead of efforts to create more choice and competition through charters, or to reconstitute the teacher workforce towards higher levels of effectiveness, or to establish high quality, intensive, and targeted preschool programs, all of which have evidence of effectiveness. It is to say that leaving curriculum reform off the table or giving it a very small place makes no sense. Let’s do what works for the kids, and let’s give particular attention to efficient and practical ways of doing so.

“We conclude that the effect sizes for curriculum are larger, more certain, and less expensive than for the Obama-favored policy levers,” writes Whitehurst, the former director of the Institute of Education Sciences.  He recommends the Administration “integrate curriculum innovation and reform into its policy framework.”

The Department of Education, through the Institute of Education Sciences, should fund many more comparative effectiveness trials of curricula and other interventions, both through its National Center for Education Evaluation and through competitive grants to university-based researchers. The Obama administration has clearly recognized the importance of comparative effectiveness research in health care reform. It is no less important in education reform.”

Can I get an amen?