School Reform Fails the Test: Part 1

This is the first half of an essay that appeared in the Winter, 2015 issue of The American Scholar. Some of you have read it already, but for those of you who haven't, I reprint it here. I will post the second half in about one week.

***

During the first wave of what would become the 30-year school reform movement that shapes education policy to this day, I visited good public school classrooms across the United States, wanting to compare the rhetoric of reform, which tended to be abstract and focused on crisis, with the daily efforts of teachers and students who were making public education work.

I identified teachers, principals, and superintendents who knew about local schools; college professors who taught teachers; parents and community activists who were involved in education. What’s going on in your area that seems promising? I asked. What are teachers talking about? Who do parents hold in esteem? In all, I interviewed and often observed in action more than 60 teachers and 25 administrators in 30-some schools. I also met many students and parents from the communities I visited. What soon became evident—and is still true today—was an intellectual and social richness that was rarely discussed in the public sphere or in the media. I tried to capture this travelogue of educational achievement in a book published in 1995 called Possible Lives: The Promise of Education in America. Twenty years later, I want to consider school reform in light of the lessons learned during that journey, and relearned in later conversations with some of these same teachers.

***

For all of the features that schools share, life inside a classroom is profoundly affected by the immediate life outside it, by the particular communities in which a school is embedded. Visiting a one-room schoolhouse in rural Montana or a crowded high school in Chicago, you will find much in the routines and the curriculum that holds steady—the grammar of schooling, as historians David Tyack and Larry Cuban called it in Tinkering Toward Utopia: A Century of Public School Reform (1995). Yet within that grammar lie differences: in topics of discussion, in the illustrations that teachers use, in how the language sounds, and in the worries of the day pressing in from the neighborhood. These differences, the differences of place, make each school distinct from every other.

During my travels, I watched as third-graders in Calexico, a California-Mexico border town, gave reports on current events in Spanish and in English. They followed the journalist’s central questions—who, what, why, when, where, and how—exploring the significance of the depleted ozone layer, of smog in nearby industrial Mexicali, of changes in the local school board.

In Chicago, 12th-graders discussed Faulkner’s As I Lay Dying, trying to make sense of the characters’ different perspectives, offering provisional explanations of important occurrences in the novel. They were gaining a sense of the power of speculation, of moving an inquiry forward by wading into uncertain waters.

On Baltimore’s West Side, first-graders combined literature and science by reading a fanciful story about hermit crabs and then conducting an experiment—resulting from a student’s question—to understand the environment in which the crabs thrive.

In small towns in the Mississippi Delta, middle school children played games with physical representations of algebraic operations, part of civil rights activist Bob Moses’s Algebra Project, a curriculum as well as a social movement that still helps prepare children, regardless of academic background, for algebra, which Moses believes is an important pathway to opportunity.

And in a one-room schoolhouse in Polaris, Montana, students kept a naturalist’s journal on the willows in the creek behind the school. At one point the teacher bent over an older student who was working on sketches and measurements. The teacher pointed to one detailed drawing and asked his student why he thought the willows grew in such dense clusters, rather than long and snaky up a tree. The boy had fished these creeks for years, the teacher later explained, and “I just wanted him to take a different look at what he already knows.”

The teachers in these varied classrooms shared a belief in their students’ ability to become engaged by ideas and to develop as thoughtful, intellectually adventurous people. They saw the subjects they taught—whether science, literature, or math—as bountiful resources that would foster their students’ development.

To update Possible Lives, I spoke to each of these teachers again about 10 years after my visit and found that all of them shared a deep concern about the potential effect of the federal No Child Left Behind Act of 2001 on the classrooms they had worked so hard to create. No Child Left Behind and the Obama administration’s 2009 Race to the Top initiative are built on the assumption that our public schools are in crisis, and that the best way to improve them is by using standardized tests (up to now only in reading and math) to rate student achievement and teacher effectiveness. Learning is defined as a rise in a standardized test score and teaching as the set of activities that lead to that score, with the curriculum tightly linked to the tests. This system demonstrates a technocratic neatness, but it doesn’t measure what goes on in the classrooms I visited. A teacher can prep students for a standardized test, get a bump in scores, and yet not be providing a very good education.

Organizing schools and creating curricula based on an assumption of wholesale failure make going to school a regimented and punitive experience. If we determine success primarily by a test score, we miss those considerable intellectual achievements that aren’t easily quantifiable. If we think about education largely in relation to economic competitiveness, then we ignore the social, moral, and aesthetic dimensions of teaching and learning. You will be hard pressed to find in federal education policy discussions of achievement that include curiosity, reflection, creativity, aesthetics, pleasure, or a willingness to take a chance, to blunder. Our understanding of teaching and learning, and of the intellectual and social development of children, becomes terribly narrow in the process.

***

            School reform is hardly a new phenomenon, and the harshest criticism of schools tends to coincide with periods of social change or economic transformation. The early decades of the 20th century—a time of rapid industrialization and mass immigration from central and southern Europe—saw a blistering attack, reminiscent of our own time. The Soviet launch of Sputnik in 1957 triggered another assault, with particular concern over math and science education. And during the 1980s, as postwar American global economic preeminence was being challenged, we saw a flurry of reports on the sorry state of education, the most notable of which, A Nation at Risk (1983), warned of “a rising tide of mediocrity that threatens our very future as a Nation and a people.”

Public education, a vast, ambitious, loosely coupled system of schools, is one of our country’s defining institutions. It is also flawed, in some respects deeply so. Unequal funding, fractious school politics, bureaucratic inertia, uneven curricula, uninspired pedagogy, and the social ills that seep into the classroom all limit the potential of our schools. The critics are right to be worried. The problem is that the criticism, fueled as it is by broader cultural anxieties, is often sweeping and indiscriminate. Critics blame the schools for problems that have many causes. And some remedies themselves create difficulties. Policymakers and educators face a challenge: how to target the problems without diminishing the achievements in our schools or undermining their purpose. The current school reform movement fails this challenge.

Back when I was visiting schools for Possible Lives, critics were presenting charts of declining scores on SATs but overlooking the demographic and economic factors that contributed to these numbers—for example, more low-income and immigrant students were taking the tests (arguably an egalitarian development). Comparing our test scores with those of other countries, the critics also failed to consider the social, economic, and cultural differences. (Students in our nation’s affluent districts fare much better in international comparisons.) The proposed remedies included not only new curricula and tests to measure the mastery of these courses of study, but also more time in school, more rigorous teacher education and credentialing, and market-based options like school choice and vouchers. And the primary goal of reform was always presented as an economic one: to prepare our young people for the world of work and to protect our nation’s position in the global economy.

Since then, the reform effort has spread and grown more intense, and it continues to focus on public school failure. No Child Left Behind and Race to the Top have dramatically increased the influence of the federal government on public schools. Both programs require states to establish standardized testing programs, and federal funding often depends on the test results. If schools don’t meet certain performance criteria, they are subject to sanction and even closure. Race to the Top added a competitive grant program to the federal effort, requiring states to lift limits on charter schools and tie teacher evaluations to students’ test scores in order to be eligible for a significant one-time award of federal funds. Some philanthropies have also supported the reform agenda, and private advocacy groups have championed causes ranging from charter schools to alternative approaches to teacher credentialing to, most recently, overturning teacher tenure and union protections.

Not all those who identify themselves as reformers would subscribe to the redefinition of teaching and learning that concerns me, and some of those reformers are raising among their peers the same issues I am. But a dominant account does emerge from many influential reform reports and organizations, and it is supported by the U.S. Department of Education.

***

A core assumption underlying No Child Left Behind is that substandard academic achievement is the result of educators’ low expectations and lack of effort. The standardized tests mandated by the act, its framers contended, hold administrators and teachers accountable—there can be no excuses for a student’s poor performance. It’s true that some teachers don’t expect much of the young people in their charge, particularly students from low-income backgrounds and underrepresented racial and ethnic groups. But because we know that so many factors contribute to student achievement, the strongest of which is parental income, the low expectations of some teachers cannot possibly account for all the disparities in academic performance. The act’s assumptions also reveal a pretty simplified notion of what motivates a teacher: raise your expectations or you’ll be punished—what a friend of mine calls the caveman theory of motivation. An even more simplistic theory of cognitive and behavioral change suggests that threats will lead to a change in beliefs about students, whether these beliefs come from prejudice or from pity. Still, No Child Left Behind’s focus on vulnerable students was important, and the law did jolt some low-performing schools into improving their students’ mastery of the basic math and reading skills measured by the tests.

But the use of such tests and the high stakes attached to them also led to other results that any student of organizational behavior could have predicted. A number of education officials manipulated the system by lowering the cutoff test scores for proficiency, or withheld from testing students who would perform poorly, or occasionally fudged the results. A dramatic example is the recent case of cheating in Atlanta, where school personnel all the way up to the superintendent were indicted.

Studies of what went on in classrooms are equally troubling and predictable. The high-stakes tests led many administrators and teachers to increase math and reading test preparation and reduce time spent on science, history, and geography. The arts were, in some cases, drastically reduced or eliminated. Aspects of math and reading that didn’t directly relate to the tests were also eliminated, even though they could have led to broader understanding and appreciation of these subjects.

Not long ago, a teacher I’ll call Priscilla contacted me with a typical story. She has been teaching for 30 years in an elementary school in a low-income community north of Los Angeles. The school’s test scores were not adequate last year, so the principal, under immense pressure from the school district, mandated for all teachers a regimented curriculum focused on basic math and literacy skills. The principal directed the teachers not to change or augment this curriculum. So now Priscilla cannot draw on her cabinets full of materials collected over the years to enliven or individualize instruction. The time spent on the new curriculum has meant trims in science and social studies. Art and music have been cut entirely. “There is no joy here,” she told me, “only admonishment.”

It makes sense to concentrate on the basics of math and reading, for they are central to success in school, and an unacceptable number of students don’t master them. And a score on a standardized test seems like a straightforward measure of mastery. But in addition to the kinds of manipulation I discussed, there are a host of procedural and technical problems in developing, scoring, and interpreting such tests. Test outcomes depend on the statistical models used, and scores can fluctuate and be marred by error—thus there is a debate among testing experts about what, finally, can be deduced from the scores about a student’s or a school’s achievement. Similar debates surround the currently popular use of “value-added” methods to determine a teacher’s effectiveness.

A further issue is that a test that includes, say, the writing of an essay, a music recital, or the performance of an experiment embodies different notions of learning and achievement than do the typical tasks on standardized tests: multiple choice items, matching, fill-ins. I have given both kinds of tests. Both have value, but they represent knowledge in different ways and require different kinds of teaching.

The nature of a school’s response to high-stakes pressure is especially pertinent for those less affluent students at the center of reform. When teachers in schools like Priscilla’s concentrate on standardized tests, students might improve their scores but receive an inadequate education. A troubling pattern in American schooling thereby continues: poor kids get a lower-tier education focused on skills and routine while students in more affluent districts get a robust and engaging school experience.

It’s important to consider how far removed standardized tests are from the cognitive give and take of the classroom. That’s one reason for the debate about whether a test score—which is, finally, a statistical abstraction—accurately measures learning. Some reform leaders, including Arne Duncan, the U.S. secretary of education, are now trying to dial down the emphasis on testing. But because tests are easy to use and have an aura of objectivity, they are likely to remain central in the reform agenda.

***

Priscilla’s story is emblematic not only of the mechanical and restrictive pedagogy that is frequently forced on teachers in a test-driven environment, but also of the attitude toward teachers. They live in a bipolar world, praised as central to students’ achievement and yet routinely condemned as the cause of low performance.

When the standardized test score is the measure of a teacher’s effectiveness, other indicators of competence are discounted. One factor is seniority—which reformers believe, not without reason, overly constrains an administrator’s hiring decisions. Another is post-baccalaureate degrees and certifications in education, a field many reformers hold in contempt. Several studies do report low correlation between experience (defined as years in the profession) and students’ test scores. Other studies find a similarly low correlation between students’ scores and teachers’ post-baccalaureate degrees and certifications. These studies lead to an absolute claim that neither experience nor schooling beyond the bachelor’s degree makes any difference.

What a remarkable assertion. Can you think of any other kind of work—from hair styling to neurosurgery—where we don’t value experience and training? If reformers had a better understanding of teaching, they might wonder whether something was amiss with the studies, which tend to deal in simple averages and define experience or training in crude ways. Experience, for example, is typically defined as years on the job, yet years in service, considered alone, don’t mean that much. A dictionary definition of experience—“activity that includes training, observation of practice, and personal participation and knowledge gained from this”—indicates the connection to competence. The teachers in Possible Lives had attended workshops and conferences, participated in professional networks, or taken classes. They experimented with their curricula and searched out ideas and materials to incorporate into their work. What people do with their time on the job becomes the foundation of expertise.

More generally, the qualities of good work—study and experimentation, the accumulation of knowledge, and refinement of skill—are thinly represented in descriptions of teacher quality, overshadowed by the simplified language of testing. In a similar vein, the long history of Western thought on education—from Plato to Septima Clark—is rarely if ever mentioned in the reform literature. History as well as experience and inquiry are replaced with a metric.

These attitudes toward experience are rooted in the technocratic-managerial ideology that drives many kinds of policy, from health care to urban planning to agriculture: the devaluing of local, craft, and experiential knowledge and the elevating of systems thinking, of finding the large economic, social, or organizational levers to pull in order to initiate change. A professor of management tells a University of California class of aspiring principals that the more they know about the particulars of instruction, the less effective they’ll be, for that nitty-gritty knowledge will blur their perception of the problem and the application of universal principles of management—as fitting for a hospital or a manufacturing plant as a school.

This dismissal of classroom knowledge fits with the trendy discourse of innovation and creative disruption, a discourse that runs throughout education reform, asserting that it will take entrepreneurial outsiders to change the system. I understand the impulse here, because getting something fresh through large school bureaucracies can be maddening. But creative disruption is predicated on the belief that anything new must be better, and it relies on a reductive model of organizational and technological change. One of the celebrated technologies in the disrupters’ armory is the computer, which clearly allows wonderful things to happen in education. But online charter schools have a troubled record, and higher education’s much ballyhooed massive open online courses, or MOOCs, are proving to be much more limited in their usefulness or success than predicted. The computer’s potential is realized only when people who are wise about teaching and learning program it, and when it is integrated into a strong curriculum taught by someone who is savvy about its use.

You can share this blog post on Facebook, Twitter, or Google Reader through the "share" function located at the top left-hand corner of the blog.