The Mismeasure of Teaching and Learning in Contemporary School Reform, Part II

This post continues the post from May 6, 2011.

Just as learning and achievement get narrowly defined in this reform world, so does teaching. Priscilla’s story is emblematic not only of the mechanical and restrictive pedagogy that is frequently laid on teachers in a test-driven environment, but also of the attitude toward teachers and the walling off of their participation and on-the-ground knowledge. Teachers today live in a bipolar world. They are praised as central to student achievement and routinely condemned as the cause of low performance. And, the overriding measure of competence or (the term used these days) effectiveness is the standardized test score in math and reading. The teacher becomes a knowledge-delivery system, and the better the students do, the more effective the teacher is judged.

No doubt, a focus on the K-12 teacher is an important feature of current school reform. Though parental income continues to be the strongest predictor of academic achievement, within the school the teacher is key. And in comparison to most of the countries (Finland, South Korea) whose educational system our policy-makers admire, teachers in the U.S. are paid less, have less substantial professional development, and enjoy less occupational status. So it could be a good thing to have the teacher’s role acknowledged on a national scale. And, as noted, some of the problems the reformers target are important ones. There is wide variation in competence in the teaching profession. How could there not be with a work force of close to four million, the largest profession in the country?

Many of these teachers – but by no means all – work within a poorly executed evaluation system, and they complain about inadequate assessment of their performance, and, therefore, inadequate guidance. One reason for this state of affairs is that principals are so awash in administrative duties that they neither have the time to conduct careful evaluations nor the training to do what their title originally signified: be the principal teacher, able to provide guidance about pedagogy. The current managerial orientation of school reform will do little to remedy that.

And sadly it is true that some teachers hold low expectations for their students, especially those who are less advantaged. I remember sitting in Elena Castro’s warm and stimulating classroom – she’s the teacher who asked her student to apply the word “admire” – when a group of teachers from a neighboring district with similar working-class demographics walked through on a visit. I heard one of the visitors whisper to another, “Our students couldn’t handle this.” Low expectations can come from out-and-out bigotry, or from jaded weariness, or from misguided sympathy – and the reformers are right to assail them.

I want to dwell on this business of low expectations for a moment longer – particularly on NCLB’s response to them, for the NCLB approach reveals a lot about the one-dimensional way teachers are understood in our reform environment.

For NCLB, it is lack of effort and low expectations that lead to low student achievement. That’s quite a claim, given all the other factors that contribute to student achievement, but let’s grant it momentarily. You’ll recall that the mechanism that will correct those low expectations is the threat of high-stakes testing. This is a pretty simplified notion of motivation: raise your expectations or you’ll be punished, what a friend of mine calls the caveman theory of motivation. And there’s an even more simplistic theory of cognitive and behavioral change at work: threat will lead to a change in beliefs, whether these beliefs come from prejudice or pity.

This kind of reductive understanding of teachers and teaching characterizes a good deal of post-NCLB reform as well. I’ll select two phenomena in current reform as illustration: the discounting of experience and the attempt to determine effective teaching practices.

For the standardized test score to be locked in as the reigning measure of teacher effectiveness, other indicators of competence need to be discounted. One is seniority – which reformers believe, not without reason, overly constrains an administrator’s hiring decisions. Another is post-baccalaureate degrees and certifications in education, a field many reformers hold in contempt. Fortunately for the reformers, there are studies that do report low correlations between experience (defined as years in the profession) and student test scores. There are also studies that report similarly low correlations between student scores and post-baccalaureate degrees and certifications. These studies lead to an absolute claim heard frequently these days that neither experience nor schooling beyond the bachelor’s degree make any difference in teacher effectiveness – and that the test score remains our only legitimate measure of competence.

On the face of it, this is a remarkable assertion. Can you think of any other profession – from hair styling to fire fighting to neurosurgery – where we wouldn’t value experience and training? If reformers had a more comprehensive understanding of teaching, they would at least consider the possibility that something is amiss with the studies. The problem is that the studies for the most part deal in simple aggregates and define experience or training in crude ways. Experience is defined as years on the job, and it’s no surprise that years alone don’t mean much. But if you define experience in one of the ways Webster suggests—“activity that includes training, observation of practice, and personal participation and knowledge gained from this”— then you would most likely find a connection between experience and competence. What people do with their time on the job is crucial, becomes the foundation of expertise. As for the question about post-baccalaureate work, the same principle applies: What kind of training? Where? What was the curriculum? The quality of supervision? I’ll be the first to admit that a number of education programs leave a lot to be desired, but to discount experience and training in blanket fashion is not only wrong-headed but also undercuts attempts to create better working conditions for teachers, more robust professional development, and opportunities for career advancement – all things the reformers say they want.

The qualities of good work—study and experimentation, the accumulation of knowledge and refinement of skill – are thinly represented in descriptions of teacher quality, overshadowed by a (often inadequately understood) language of testing. In a similar vein, it is telling that the long history of Western thought on education – Plato to Septima Clark – is rarely if ever mentioned in the reform literature. History as well as experience and inquiry are replaced with a technological metric.

The cases of experience and training provide an example of the way many of the reformers’ foundational assumptions about teaching and learning are constructed. There is much made about science in current reform, talk of “data-driven decision making” and “evidence-based practice.” But some policy decisions (like the expansion of charter schools) are made without much research support, and bedrock claims such as “experience and training don’t matter” are based on narrowly conceived and executed analyses.

***

Last year there was a long article in the New York Times Magazine about attempts to pinpoint the techniques that comprise good teaching—from standing still when giving directions to ways to pose a math problem and call on students—and teaching those techniques to teachers. The article was titled “Building a Better Teacher,” and that title captured for me both the promise and the potential limitations of a powerful strand in current school reform: A technique-and-testing orientation to improving teacher quality.

Teaching is a complex activity that, when done well, requires, among other qualities, subject matter knowledge as well as skill in teaching it. Teaching requires a knowledge of young people’s minds and hearts and, more particularly, a knowledge of one’s students and the ability to read them, read the dynamics of the room, and react appropriately. The good teacher holds a belief in human ability and a commitment to fostering it—and one manifestation of that belief is the creation of a safe and respectful as well as demanding classroom. Effective techniques are an important part of this mix, and good mentorship includes a close analysis of what a teacher is doing—like a coach and athlete watching a film—and providing corrective feedback and better ways to do things. Contrary to the story reformers tell, teachers of teachers have been doing this for a long time. What is new is the strong focus on techniques, the increased role of electronic and testing technology to study them, and the attempt to define “effective” by seeking positive correlations between specific techniques or clusters of techniques and, you guessed it, students’ standardized test scores. What is also new is the magnitude of the effort, punched up considerably by a 45 million dollar project funded by the Gates Foundation. (I am involved in another Gates-funded project.) There is, indeed, a considerable push among some reformers to build a better teacher.

Because teaching does involve a good deal of craft, I’m sympathetic to this attempt to analyze useful techniques and make them available to new and developing teachers. But given the technocratic orientation of contemporary school reform, I worry that other aspects of teaching less easily observed and circumscribed—from bearing and pacing to beliefs about learning—will get short shrift. The building of effective teachers will occur through the accumulation of techniques. And given the need in reform-initiated research to find correlations between techniques and test scores, researchers will veer toward those techniques that are most readily definable, leading to a possible narrowing of the repertoire of techniques themselves.

There is a further issue: The use of any technique gains meaning in a time and place. Consider the Chicago math teacher Michelle Smith’s decision to move her student from the back to the front of the room. She’s rearranging seating to insure order in the room, an activity that could be considered an effective technique for classroom management. Imagine, however, the other unpleasant ways that decision could have played out: The student refusing to move, or insulting or threatening her, or stirring up his comrades sitting nearby. But Michelle’s action occurs in the context of a relationship with the class and with that boy, a legacy of care and learning. (“Miss Smith,” he later told me, “she’s teaching us how to do things we couldn’t do before.”) Michelle knows local culture, understands rituals of masculinity and the huge importance of allowing that boy a little space to save face. She has developed a classroom persona that blends sass and seriousness and uses it strategically. Technique is contextual and part of a performative flow of events.

When you focus on technique without regard to context, you can get analyses like the following, taken from a New York Times column on the Gates project. Two researchers are rating the videotape of a teacher they don’t know. They zero in on a segment where she doesn’t see or ignores a boy who is raising his hand repeatedly. The teacher gets a low mark on “respect and rapport.” That’s a legitimate possible rating. But what if that boy frequently takes up conversational space, and the teacher has spoken with him about it, explaining that she can’t always call on him. I and other teachers I know have done this. Then that teacher’s actions would be seen in a different light—demonstrating a potential error in rating.

A further level of error can occur as we move to the center of this machine, finding what practices correlate with test scores, for the scores themselves are typically not stable. (In value-added measures, for example, a significant number of teachers who are in the top quartile of scores one year will not be in the top the next; the same holds for the bottom quartile.) And finally there is the attempt to correlate practice to test scores which, precedent for this kind of study suggests, could yield a slew of inconsequential correlations. The research design is mechanistic and reductive—it segments human activity—and is complicated with layers of potential error.

But even if you grant my concerns, isn’t it worthwhile to at least call attention to the tricks of the trade that good teachers use and make those widely available? Absolutely. And here is where we run into perhaps the most considerable problem for reformers as they turn their attention to teacher development.

The entire history of this reform movement has not been built on teacher development but rather on a punitive accountability system of high-stakes testing. (Recall the NCLB feet-to-the-fire approach to motivating teachers.) Lip service was paid to helping teachers use their students’ scores to improve instruction, but little broad-scale infrastructure was put in place to enable even that kind of development to happen. For the current program to be successful, there will need to be a sea-change in attitudes toward and understanding of teachers and the teaching profession. Now would be the time to start.

Though I am disheartened by how few reformers express a robust vision of public education, there is no doubt that they are committed to education itself, and particularly for those who have not been well-served by our schools. But to get the kind of teaching we saw in the opening vignettes, to make more of our schools the potent democratic institutions they can and should be, the reform movement will have to generate within itself a much richer sense of teaching and learning. We have a strong belief in our country that to find a measure for something is to understand it; we confuse counting with analysis. Education reform needs a conceptual framework that would certainly include testing and technique but embedded in the cognitive and emotional world of the classroom.

I would like to thank Megan Franke, Judy Johnson, Felipe Martinez, John Rogers, Shirin Vossoughi, Noreen Webb, and Kyo Yamashiro for their help with this article.