Wednesday, January 22, 2014

Thoughts on Educating Teachers When Teacher Education Is Under Attack: Part Three

This is the third and last of reflections on teacher education programs. It appeared in Valerie Strauss's "Answer Sheet" column in The Washington Post on January 13, 2014.

A number of people provided information and advice for this series, particularly with the more technical aspects of the analysis. Thanks to Tina Christie, Lisa Dillman, Yrjo Engestrom, Megan Franke, Kris Gutierrez, Felipe Martinez, Ted Mitchell, Aaron Pallas, Jody Prisliac, Rema Reynolds, Glory Tobiason, Noreen Webb, and LuAnn Wilkerson. 

***

            College and university-based teacher education programs vary considerably by size, region, student body, nature and focus of curriculum, talent of instructional staff, status within home institution, balance of coursework and practice, relation with local district, and more. Some are excellent, some are good and experimenting with ways to get better, some are weak in some respects but decent in others, some are marginal and poorly run. The language of the current criticism of teacher ed, at least the most public language, doesn’t allow for this variability. Nor does the dismissive rhetorical stance of the most vocal critics, the tone and attitude running through their language. The bottom-line message: Teacher education is a disaster.

            I understand the use of heated language to get the public’s and policy makers’ attention; in that regard it is rhetorically effective. And I can also understand—and certainly have felt—the exasperation with the slow pace of change that can lead to such language. But, as I’ve argued before about the rhetoric of school reform generally, a sweeping language of failure narrows the understanding we have of a problem and leads to solutions that create problems of their own.

            Let me provide examples from the two reports I’ve cited in my previous posts.

***
The Language and Rhetorical Frame of the Reports

            The 2005 report by Arthur Levine, Educating School Teachers, provides an example of the way an amped-up language of failure misrepresents the current state of teacher education. In the preface to the full report, Levine writes that though he was recently the president of Columbia University’s Teachers’ College, Educating School Teachers is not a “defense” of teacher ed programs, nor is it the “attack” that critics hoped for: “The aim is to let the data speak for themselves and to allow the chips to fall where they may.” This is an admirable position, but the rhetorical frame and language Levine chooses undercut his goal. Here is a sampling of the section titles: “The Pursuit of Irrelevance,” “Inadequate Preparation,” “A Curriculum in Disarray,” “A Disconnected Faculty.” Sounds awful, right?

            When you read these sections, however, what you often find is that the institutional, demographic, and survey data Levine draws on reveals the kinds of negative findings that would support the language of failure about 30 to 50 percent of the time. The other 50 to 70 percent of the data get much less attention and analysis. In a nine-paragraph subsection of “A Disconnected Faculty,” for example, there is one sentence that quotes a positive comment about teacher ed faculty’s connection to the schools, and at the end of the discussion a brief mention of the four model programs described in the conclusion of the report. Yet the survey data presented in the subsection reveals that 30 to 45 percent of teacher ed alumni agree that “faculty are not sufficiently involved with local schools.” The other 55 to 70 percent of more positive alumni responses and comments are barely acknowledged. The Executive Summary, which is the document all but the most thorough journalists read—if anything beyond the press release is read at all—makes no mention of the distribution of the data.

            Arthur Levine is a major figure in higher education scholarship and policy—I’ve admired and used his research on the history of the undergraduate curriculum—so one wonders what is going on with this reporting of data. I don’t know Dr. Levine, so this is conjecture, but I suspect that he set out to grab the attention of his colleagues nationwide and jolt them into action. Though he recommends closing down a number of teacher ed programs, he is not writing a brief for alternative credentialing programs “which offer far less preparation prior to entering a classroom.” Educating School Teachers is a jeremiad, an angry cry to fellow higher education scholars and administrators to change in order to thrive. Unfortunately, the rhetorical purpose of the report, I think, leads to, almost commits one to, a one-sided representation of data, and it ends up providing powerful ammunition for those who have goals quite different from Levine’s. As we’ve seen in K-12 policy, the language of failure takes on a life of its own.

            There is no need to speculate about the stance of Kate Walsh, the president of the National Council for Teacher Quality, for since her days as a senior policy analyst at the Abell Foundation in Baltimore, she has been a fierce critic of teacher education and an advocate for alternative credentialing programs. (Consider the title of a report she wrote in 2001: Teacher Certification Reconsidered: Stumbling for Quality.) If Arthur Levine’s report is a jeremiad, NCTQ’s is a polemic that in a rhetorically effective way draws on the conventions of the research report.

            Much has been written about the problems with this report, particularly about the significant limitations of its analysis, built primarily on one kind of information: syllabi, course descriptions, and other program materials. Because of NCTQ’s well-known animus toward teacher ed programs, only a small number of programs willfully complied with requests for this information, so the Center filed open-records requests, litigated where it could, searched the Internet, queried students and districts, and so on—setting up a contentious dynamic that suffuses Teacher Prep Review. At several points, the authors appeal directly to readers to pressure their institutions to comply with NCTQ. The gloves are off.

            The authors of Teacher Prep Review make the case for the legitimacy of their analysis through a language of science. They stress the rigor of their procedures, the many steps involved, the use of a technical panel and an audit panel—and the report is thick with tables and charts, further communicating technical sophistication. But as I’ve argued in previous posts in this series, a study can be technically sound but limited. Some ed school administrators have complained that they have informed NCTQ about errors in the Center’s analysis, but to no avail. Still, let’s assume that the analysts at NCTQ were methodical and detailed in their examination of the materials they had. The problem is that what they had provides a narrow slice of what comprises a teacher’s education. Good science is good not only at the technical level, but at the conceptual level as well—and the technical and conceptual deeply intertwine. It is at the conceptual level—the level of understanding of teaching and teacher education and how one might study them—that Teacher Prep Review falls short.

            Without observing and interviewing new teachers to get a sense of the fuller scope of their educational experience, we can’t know what they did or didn’t learn about, to use the report’s examples, teaching reading or managing a classroom. (Teacher Prep Review has sidebar quotations—all negative—from five teachers and two principals. These quotations could be from a survey, but there is no presentation of the results of that survey, and I could find no mention of it in the methodology section.) We are dealing here with the basic issue of epistemology, how we come to know something. A related and fundamental issue is our assumptions about what the something is. I’ll have more to say about this shortly, but for now, let me pose a question: Would a reader of Teacher Prep Review—or, for that matter, the authors of the document—want any entity that matters to them (their business or professional institution, their church, their kids’ school) evaluated by one criterion only, by one kind of information?

            The second conceptual issue to consider is the fact that the authors have a strong point of view about what should and shouldn’t be taught in teacher education—and how content should be taught. In the case of Ms. Walsh, that point of view predates the production of this report. It is certainly the authors’ right to have a strong perspective, but it needs to be acknowledged as a potential source of bias, for it influences their research and the rhetorical frame in which they present that research. Rather than being an objective report on the current state of teacher education in the United States, Teacher Prep Review becomes an argument for a particular kind of teacher education and, de facto, for a particular definition of teaching.

            The authors characterize the entire field of teacher education as eschewing training and practical advice, and instead favoring a curriculum oriented toward exploration of novice teachers’ “prejudices…related to race, class, language and culture” and the development of the “professional identities of teachers.” Though this characterization is sweeping, the authors note that what they’ve done is find “programs throughout the country bucking the reigning ethos and actually training their candidates in crucial skills.” They view the publication of Teacher Prep Review as a “turning point,” for “the consumers of teacher preparation—aspiring teachers and districts—at last have the information they need to choose what programs to patronize. Collectively, their choices will shift the market toward programs that make training a priority.”

            As I’ve been writing throughout this series, my intention is not to defend traditional teacher ed programs as a whole. Some of the issues raised by NCTQ—early reading, classroom management, preparation for Common Core Standards—are significant ones, and it is surely the case that some programs do better with them than others. But what concerns me is the flawed model, the incomplete template being offered as to how we might explore these issues. Of equal concern is the cloaking of ideology in the objectivist language of science.

            There is an important passage toward the end of the methodology section of Teacher Prep Review where the authors list the limitations of their work: “It is not the intention of Teacher Prep Review to substitute for high-quality, on-the-ground inspections as one might expect an accrediting body or government authority to perform…We restrict our evaluation to only program elements that can be reliably and validly assessed by readily obtained program documents.” This is an accurate statement of limitations that the authors violate on a grand scale, for they use their self-confessed limited analysis to condemn hundreds of programs—with the stated intention of driving them out of business. Hundreds of other programs are labeled “weak,” and “someone who wants to become a teacher would be better off investing time and tuition dollars elsewhere.” Rather than a restricted evaluation, Teacher Prep Review becomes an activist polemic.

***
The Potential Effect on Teaching

            I want to bring these posts on teacher education to a close by returning to my initial discussion of teaching, what it is, and how it might be affected by the current criticisms of and proposals for teacher ed.

            Though I am concerned about the possible negative consequences of the way the criticism is delivered by the high-profile critics, I want to make clear that they raise some important issues for the education of teachers in traditional or alternative programs and, therefore, for the students these teachers will encounter. It should be noted that people within ed schools have been raising these issues as well. Does a particular program offer courses that are relevant to the work novice teachers will soon be doing? Does that program strike the right balance between coursework and work in the field, and are the two connected in a generative way? And does the program have a systematic way to follow up on its graduates and incorporate what it finds back into its course of study? The National Academy of Education recently released a report, Evaluation of Teacher Preparation Programs, that provides detailed guidelines on conducting an evaluation that could answer these and other relevant questions.

            One could grant my concerns about the critics but argue that they (and the alternative programs some of them lead) provide a necessary counterweight to the lapses or excesses of traditional teacher education. Fair enough. My worry though is that the correction they provide takes us too far in a reverse direction, replicating a troubling pattern in American education of pendulum swings from one pole to its opposite. One potential swing that I address in a previous post is the emphasis on practice with a discounting of coursework in the history and philosophy of education, theories of learning and child development, and the like.

A variation of this polar swing is a focus on teaching skills and techniques over more philosophical and social-cultural topics. This tendency is illustrated in Doug Lemov’s Teach Like a Champion, a distillation of forty-nine techniques, from bringing a classroom to order, to asking questions, to correcting bad behavior. Teach Like a Champion is endorsed by some of the major players in the alternative credentialing movement, and Lemov was a member of the National Council on Teacher Quality’s technical panel. The book’s value lies in the detail with which Lemov describes the techniques, down to counting the seconds saved by phrasing a question one way rather than another. Though I would want to modify some of the techniques and augment the behavior management approach that informs them, I think Teach Like a Champion is a useful and thought-provoking taxonomy (Lemov’s word) of pedagogical techniques for classroom practice. The problem with the book to my mind is that it reaches beyond its taxonomy to become a philosophical and definitional statement. Lemov ends up equating good teaching with technique. “Artists, athletes, musicians, surgeons…achieve greatness only by their attention to the details of their technique…This focus on technique and its constant refinement is also the path to excellence for teachers.” Furthermore, the techniques, Lemov claims, have a direct causal link to achievement and “put students on the path to college.”

            Techniques are vitally important, but don’t work in isolation. The sequencing of questions, for example, is a crucial skill, but is dependent on the teacher’s knowledge of the material being taught and knowledge of how children typically respond to it, the kinds of misconceptions and errors they make, the alternative explanations, illustrations, metaphors and analogies that might help them. A teacher can’t ask meaningful questions for long without this kind of knowledge. In equal measure, the effectiveness of techniques, particularly for classroom management, will be influenced by students’ sense of a teacher’s concern for them and understanding of them. The touchstone of school reform for over a decade has been the need for high expectations for all students. If we’re serious about addressing “the soft bigotry of low expectations,” then there has to be room in teacher education for novice teachers to investigate and reflect on the limiting beliefs about cognition and ability that they, that all of us, inhale in the cultural air we breathe.

            To be sure, the quality of teacher ed courses on the sociology of schooling, on culture, on linguistic diversity, and the like vary widely, from the imaginative and dynamic to the routine and disconnected. What I am arguing for is the place in teacher preparation, traditional or alternative, for sociological and cultural topics and for the occasion for philosophical reflection on this complex value-laden work new teachers are undertaking. To teach well requires, among other skills and bodies of knowledge, the intellectual tools to understand who your students are, where they live, the history that precedes them and shapes them. Put another way, will we help teachers develop the acumen to analyze what might be going on when their techniques, no matter how refined, don’t work?

            Kate Walsh of NCTQ and like-minded critics pose an either/or question: Would parents rather beginning teachers have a class in teaching skills or classroom management or take a class on sociological or cultural topics? Imagine a different question in similar form: Would you rather have your child in a classroom that is well-managed or a classroom that conveys an understanding of your child and that fosters his or her engagement in learning? My guess is that most parents of any demographic category would say they want all these qualities, for in the good classroom all are interrelated.

            There is one more issue that emerges in the teacher ed debates that is worrisome: the relation of poverty to academic achievement. Let me again go to Teach Like a Champion for an illustration. In the introduction, Lemov reflects on the teachers he’s observed to develop the taxonomy of techniques: “These outstanding teachers routinely do what a thousand hand-wringing social programs have found impossible: close the achievement gap between rich and poor, transform students at risk of failure into achievers and believers, and rewrite the equation of opportunity.” The passage illustrates a tendency among some teacher ed critics that we see in larger school reform debates: to pose schooling as the solution to poverty, the intervention that will work where other interventions fail. Here again we have the reinscribing of a binary: school versus social programs. (There is also an affirmation of technocratic solutions—in this case, technique-driven teaching—to social problems.)

            About fifteen pages later, however, Lemov offers a moving anecdote that serves as a reminder of the ugly staying power of inequality. He is telling the story of a former student of his, “the bright and passionate son of a single mother with limited English” who made the remarkable journey to Williams College. At college, though, the student’s problems with writing dogged him and were reflected in a professor’s unfavorable response to a paper he wrote on Zora Neale Hurston. Lemov tells this story to stress the importance of teaching students standard written English. But in my eyes, having worked in university programs that serve students like this one—and having been such a student myself—this story represents the intractability of inequality, that after the best teaching Lemov and his colleagues could provide, this young man still needed assistance at further points along the way. One hopes that Williams had the wisdom to provide it. The student will also need people who understand what he must be feeling, the crushing disappointment, possibly anger, and the deep cut to his confidence. Schools like Lemov’s might be able to affect an achievement gap, the scores on district or state standardized tests, but not necessarily single-handedly erase the achievement gap, which requires sustained help of many kinds, including programs that Lemov dismisses as “hand-wringing.”

***

            The current incarnation of school reform has been with us for over a decade, and not only its detractors but its supporters as well acknowledge its unintended consequences. The recent criticism of schools of education emerges from the same reform principles and techniques, and I’ve tried to tease out in my three posts some of the potential unintended consequences for teaching and teacher education that could result from this criticism.

            There is a boldness to the criticism and an entrepreneurial can-do spirit to the high-profile alternative credentialing efforts that is appealing to Americans. The goal is to improve our schools, close the achievement gap, and restore opportunity and mobility. Powerful and laudable. But the criticism has flaws in it that should instill in us a little caution, not to forego improvement of teacher education and develop new ways to provide it, but rather to help us move forward on surer footing. The criticism sometimes includes big claims based on hasty cross-cultural or cross-institutional comparisons, on statistics that are picked out of larger, more varied data sets, and on causal claims that are not empirically supported. A language of science (“research based,” “evidence based”) suffuses the criticism, but, at times, is not warranted by the facts or analytic procedures underlying the language. Though some of the critics claim to be above ideology, basic assumptions about learning, motivation, and the goals of education drive their arguments—assumptions that might well have merit, but need to be clearly articulated and investigated.

            Finally, all I’m asking is that we be a little more discerning in language and claims and not repeat past mistakes or stumble into new ones as we educate our next generation of America’s teachers.

You can share this blog post on Facebook, Twitter, or Google Reader through the "share" function located at the top left-hand corner of the blog.  

Monday, January 6, 2014

Thoughts on Educating Teachers When Teacher Education Is Under Attack: Part Two

This is the second of three reflections on teacher education programs during a time when they are under attack. It appeared in Valerie Strauss's "Answer Sheet" column in The Washington Post on December 16, 2013.

***

There are many different types of effective teaching and many roads to get there. Travelling across the United States to document good public school classrooms forPossible Lives, I saw solid to extraordinary teachers of many stripes: shy and outgoing; desks in rows and desks all over the place; some were low-key and methodical, and some were energetic and spontaneous; some swore by one way of organizing their curricula and classrooms that others would find unworkable; some spoke a fair amount, others turned the floor continually back over to their students.

Yet within the variability, there were qualities they all shared. They had command of the material they taught. They created safe and respectful classrooms. They had a deep belief in the ability of their students and held high expectations for them. They required their students to think and think hard and worked to engage students in each others’ thinking. The richness came in the variety of ways they realized these qualities—an important point, given the push by some for increasingly regulated curriculum and pedagogy.

Part of the variation, of course, was a result of where these teachers went to college. But the variation also came from influential teachers they had earlier in their own schooling. The way they taught was also influenced by their personalities and by their values and background: by family or religion or positive or negative experiences in school; by the experience of race or ethnicity, social class, gender, or sexual orientation; by political and social commitments; by the love of a subject. An important quality of a teacher education program, traditional or alternative, is how well it is able to draw on and develop these characteristics. You won’t see this quality mentioned in any of the high-profile reports on teacher education.

With a few exceptions, the teachers in Possible Lives came from modest middle-class to working-class backgrounds. (This tends to be true for teachers as a whole.) A fair number went through local or regional teacher ed programs—the kinds of programs that have been targeted in teacher ed critiques. Because of finances or family expectations or cultural norms, some of the teachers I observed had few other options.

            One compelling reason behind the rise of alternative credentialing programs is to draw into the teaching force a wider sweep of people from a range of backgrounds—particularly people who might not otherwise have gone into teaching. This is all to the good. But at the same time there’s this expansive impulse in the discussion and debates around teacher ed, there is also a restrictive counter-force: calls to raise admissions standards into teacher education and recommendations to limit or close particular kinds of teacher ed programs. Let me consider each in turn.

***
Raising Standards

The general complaint here is that traditional teacher education candidates, on average, come from the lower ranks of their class and score below the national average on SAT and GRE exams. There is more variation here than the average suggests, however: some local colleges take many of their applicants, and some universities are quite selective. Also secondary-level candidates tend to have higher grade point averages and test scores than their elementary-level peers. Still it is of course true that we want to do everything possible to draw people with strong educations into teaching. Teaching is intellectual work, as I noted earlier, and I think that one thing that has limited the profession is that teaching—especially at the elementary level—is not typically defined that way. Teachers need a good general education, and, hugely important, I think, they need to be interested in education, gain pleasure from learning and thinking about learning. For those who will teach a particular subject, they need to be well educated in that field.

So, to be clear, knowledge matters. I have seen too many instances of teachers providing superficial or downright incorrect comments on student papers or stumbling through a science or math lesson on material they clearly don’t understand. But knowing something, as fundamental as that is, is half the story; knowing how to teach it—“pedagogical content knowledge,” in psychologist Lee Shulman’s famous phrase—is equally important. We have a history in the United States of defining teaching primarily in terms of process and technique or in terms of subject matter knowledge. Of the many fruitless dichotomies that bedevil education, this is among the most unproductive.

One of my concerns about the contemporary teacher ed debates is that knowledge—as represented by undergraduate major and GPA—is held in some circles as the touchstone of teaching excellence. Certainly a big part of Teach For America’s appeal is the undergraduate pedigree of its interns. And it seems to be the hope of some alternative programs that if we just get more “smart” people—smart defined by academic background, GPA, test scores—into teaching, we will have gone a long way toward solving the “teacher quality” issue. But an undergraduate at our most prestigious colleges and universities can go through four intense years of literature or chemistry and never once be confronted with the question: How would I teach this?

A while back, I spent time doing research in a top-ranked medical school. To a person, the students had through-the-roof academic credentials and did exceedingly well in their first two years of science courses. Talk about smart! The striking thing was that a fair number of them had real difficulty as they moved toward patient care. Not only were they socially inept—distant, awkward—but also diagnostically maladroit, partly because they couldn’t communicate with their patients and partly because of the difference between knowing physiology and using it to diagnose and help cure another human being. In response to this not uncommon state of affairs, medical schools across the country have been modifying supervision; instituting courses in communication, patient care, “doctoring,” and the art of medicine; and changing their recruiting and admissions policies to widen the net, gaining some students who might not have the same astronomical GPAs, but possess other qualities that contribute to being a good doctor.

I think we need to be cautious about conflating academic achievement with the ability to teach. The two are intimately related, but not one and the same.

There is a further issue, and that is the diversity of the teaching force. What happens to our talent pool as we tighten restrictions on who gets into teacher education programs? Who might get left out? Some of the young people who are most passionate about teaching in low-income communities come from those communities, and therefore have probably not had either the in-school or out-of-school resources that contribute to strong post-secondary achievement—particularly for certain majors. This scenario does not hold true for all students coming out of low-income schools, but for enough to concern us here. Passion alone does not warrant entry into the teaching profession by any means. If our candidates still need to further develop their academic knowledge and skills in certain areas, then they must do so before or during their teacher ed program—and the program needs to hold them accountable. But to systematically exclude them in a country so beset by structural inequalities is to bar from the classroom a group of people most familiar with the barriers low-income students face and deeply committed to helping those students get a better education than they did.

One last point. Another argument in the air for raising admissions is that a higher entrance bar will enhance the status of the teaching profession. Countries such asFinland are invoked where teachers face tough entrance criteria and enjoy solid professional status. These kinds of claims, and the invoking of other countries to support them, reveal one of the problems in the teacher ed debates: a tendency to make simplified causal connections and cross-cultural comparisons. Reading the sociological scholarship on the development of professions reveals what a complex process professionalization is—influenced by cultural traditions, politics and economics, gender and racial dynamics, the role of advocacy organizations and powerful leaders, and more. And the way these factors played out for teaching over the last century in the United States and Finland are pretty different.

Raising teacher ed entrance requirements in our country might have some effect on occupational prestige, but it would be one of many factors determining status, more potent ones being salary, gender bias, and degree of occupational autonomy. There may well be good reasons for a particular teacher education program or group of programs to raise its admissions standards, but that decision would need to be made after careful analysis of potential benefits and liabilities for its region and not on simplistic sociological abstractions.

***
Closing Teacher Education Programs

            Though programs in all types of colleges and universities come in for criticism in the major teacher education reports, those housed in less prestigious institutions that produce over fifty percent of our teachers—regional state universities, small public and non-selective private colleges—take a hard hit.

Consider two reports that got a good deal of media attention: Arthur Levine’s 2005Educating School Teachers (mentioned in my first post) and Teacher Prep Review,released in 2013 by the National Council on Teacher Quality (NCTQ). Though both reports single out a (quite small) number of good programs—and those programs range in type and size—the overall assessment they present is devastating.

The Levine report recommends closing poor programs, many of which, he believes, are in those state universities and non-selective colleges. In turn, the programs that should be expanded are located in research universities. (This same advocacy for research university programs runs through other, earlier reports, such as that by the 1980s Holmes Group.)

The National Council on Teacher Quality was able to rate 1,200 programs, and it placed about 15% of them so low as to warrant a “consumer alert.” Many of the 15 percent, though not all, are the same kinds of programs Levine criticizes. The authors of the NCTQ report hope that the warning will lead prospective teachers to vote with their feet and go to other schools (and school administrators to look elsewhere for new hires), thus forcing the targeted schools to improve or go out of business.

            It is not at all my purpose here to defend poor programs, or even to dispute the possibility that, on average, sub-par programs might be found more in one category of institution than another. But I do want to raise several concerns.

There’s an assumption in some of the reports—clearly stated in the one from NCTQ—that students interested in a teaching career are free agents, able to make the classical economists’ rational choice about benefits and losses, and act accordingly. They are able to go to the school that will provide the greatest payoff. But, as I noted earlier, some students are not in a financial or personal position to make such a choice. The local teacher ed program is their only option. Reading these reports, one gets the sense that the authors are at a great social distance from the lives of such students.

Some of the reports also operate at a real distance from the colleges and universities they criticize. What struck me about several of the small out-of-the-way programs I visited during my travel for Possible Lives was how embedded they were in their communities, how well the faculty understood the kids in the schools, the local history, the social and economic pressures on the region. Some of the faculty themselves went to local, non-elite colleges or universities, they didn’t publish in scholarly journals, they didn’t have the bonafides of their contemporaries in snazzier institutions. But they were smart and skillful, and they provided substantial support to the novice teachers in their charge: mentoring them, meeting with them after hours, observing them teach.

These were two good programs, and I bring them up not to generalize from them, but to illustrate a point about analysis from a distance. The National Council on Teacher Quality report could not get to the qualities I sketched. It is built primarily on analysis of course descriptions and syllabi. These will provide course philosophy and purpose in an abbreviated form, reading lists and topics, assignments, grading criteria, and the like. Little more. The Levine report utilized a much more comprehensive methodology: surveys of principals and education school administrators, faculty, and alumni; site visits to twenty-eight ed schools; and statistical analysis of data on program graduates and the students in the schools where they teach. The site visits focused on institutional structure, governance, and demographics, but I was not able to tell from the report if the visits also got to the more experiential level that I raise here.
The Levine report, as substantial as it was, raised other concerns about policy recommendations for categories of institutions. Let me provide one example, for it represents a kind of reasoning we see all too often in current education debates.

Levine commissioned a study to compare the reading and math scores of students by the type of teacher ed program their teachers attended. The statistically significant results demonstrated that students who were taught by teachers who attended research universities showed one-and-one-half weeks more growth in math than students taught by teachers who attended the aforementioned less-selective institutions. “Over the course of 12 years of schooling,” Levine writes, “this amounts to four and a half months” of growth. A result like this gets shortened in debate and opinion pieces to damning evidence that a whole slew of teacher ed programs produce poorly trained teachers. Let’s consider this result, and the reasoning that leads from it to a significant policy recommendation.

It’s important to remember that, though ambitious, this is a single study that would need to be replicated. Furthermore, the difference of one-and-one-half weeks of growth over a school year is in fact a small difference or “effect size,” and it gains statistical significance because of the large numbers of students and teachers in the sample. Effect size is a basic issue in such analyses: one can run a technically flawless analysis with a large sample size and get a statistically significant result, but the important question is whether that result matters enough to lead to a decision to act—in this instance to build a case for closing or scaling back a group of teacher ed programs.

There is also a logical problem at the heart of this example. Levine extrapolates from a single one-year study and projects out over 12 years. (Let’s put aside for a moment my contention that the effect size here is not alarming—nor, following Levine, is four-and-one-half months over a 12 year period.) For the score differential found in one year to maintain itself over 12 years requires that all other factors in the lives of the children and their schools remain the same: that the students maintain the same level of motivation, don’t get sick, don’t experience family disruption. That teachers are equally immune from life’s perturbations, and when that is not the case, they are quickly replaced. That the school-level leadership doesn’t change; that new policies aren’t enacted; that funding remains stable; that the community isn’t hit with economic hardship; and so on. The 12-year extrapolation assumes an “other things being equal” statistical model in a world where very little remains equal. Such extrapolations make for dramatic statements, but they are not conceptually sound and should not be part of the logic of a policy recommendation that would have serious consequences for many regions of the country.

            As with any institution, there is a range of quality in teacher education programs, and some are beset by the kinds of problems Levine and others identify: poor leadership, a fragmented curriculum, inadequate opportunity for students to engage in classroom practice. If in fact more troubled programs exist in the category of institution that produces over one-half of our teachers, then one would think that an important educational and social agenda would be to focus on ways to help them improve where possible and not to advocate for their abandonment and closure.

You can share this blog post on Facebook, Twitter, or Google Reader through the "share" function located at the top left-hand corner of the blog.