A Brief Reflection on Secretary of Education Miguel Cardona and the Use of Data

It is the evening of April 1, and I am listening to “1-A,” the excellent public affairs show that comes out of WAMU in Washington, D.C. Secretary of Education Miguel Cardona is the guest, and he is explaining to host Jenn White the rationale for his recent decision to mandate standardized testing for this school year. I had high hopes for this man, given all his experience in schools, and I still do, but what he is saying makes my heart sink. In essence, the more data we have, the better able we will be to distribute funds from The American Rescue Plan to the schools that most need them. He then insists that determining need will be the only use of the data, that scores will not be used to evaluate schools or teachers, to stigmatize or to punish.

         There are a number of arguments against giving the tests this year, from logistical and methodological problems to the added stress on school personnel and students during an already stressful time. These and other arguments were unsuccessfully made by hundreds of education experts in a formal appeal to Secretary Cardona. For a thoughtful analysis of why Cardona’s decision is misguided, see Jan Resseger’s blogs of March 29 and April 2.

         In addition to all else that’s been written about the folly of testing, I simply want to reflect briefly on several points made by the Secretary on “1-A”—the things he said that led to the aforementioned dropping of my heart.

         First, in what fantastical world does the Secretary think that this year’s test scores will not be used for any purpose beyond the allocation of funds… and that his Department’s pronouncements alone will function as a magic shield against abuse? Miguel Cardona has been in high levels of educational administration for a long time. He is a political animal. He surely knows better.

         Second, and to me more troubling, is the way the Secretary seems to regard “data” and the quantity of data as an automatic good, as unquestionably worth pursuing. I too believe that gaining more data is generally beneficial for everything from life decisions to policy formation, but one has to consider the quality of the data one is collecting and the context in which it is collected. One also has to ask at what cost the data are collected. Administrators, teachers, students, and parents are facing unprecedented challenges as it is without the added burden of a full-scale, high-stakes testing program, a burden that will be strongly felt in the very schools Cardona says he is most concerned about. The Secretary says that we need this data to pinpoint those in greatest need. My God, as if we don’t already know this! We have multiple measures of educational and economic inequity and need. Few physicians would approve of an invasive and destabilizing test that might yield one small bit of additional data about a patient’s well-established condition.

         I’m left with the worry that our Secretary of Education, for all his gifts, holds a technocratic faith in measures, numbers, data points—a faith that numbers, and more numbers, are always beneficial, overriding considerations of the human cost of collecting certain kinds of data and the circumstances in which they are collected.

You can share this blog post on Facebook, Twitter, or Google Reader through the "share" function located at the top left-hand corner of the blog.

Portraits of Thinking: A Novice Cabinetmaker

Here is a second story about cognition in action, a glimpse at a young man developing skills as a cabinetmaker. For those of you who missed the previous entry where I discuss the purpose of these portraits of thinking, I’ll repeat two introductory paragraphs now. If you did read the earlier entry, you can skip right to the story of Felipe, which is drawn from The Mind at Work.

As I’ve been arguing during the year of this blog’s existence—and for some time before—we tend to think too narrowly about intelligence, and that narrow thinking has affected the way we judge each other, organize work, and define ability and achievement in school. We miss so much.

I hope that the portraits I offer over the next few months illustrate the majesty and surprise of intelligence, its varied manifestations, its subtlety and nuance. The play of mind around us. I hope that collectively the portraits help us think in a richer way about teaching, learning, achievement, and the purpose of education—a richer way than that found in our current national policy and political discourse about school.

***

Felipe, a student in a high school wood construction class, is the head of a team assigned to build a cabinet for his school’s main office. At this point in his education Felipe has built one small, structurally simple cabinet, and this current, second, cabinet has a number of features the earlier project didn’t. His storehouse of knowledge about cabinets, his “cabinet sense”, is just developing, and the limits of his knowledge reveal themselves at various points throughout assembly. Like this one.

Felipe is trying to record final figures for all the components of the cabinet—he and his co-workers are eager to begin assembly. He is working with Gloria and Jesus, and he is sketching with them one more three-dimensional representation of the cabinet, using several lists and a sketch he and the others had produced during planning.

When I approach the team, Felipe is looking back and forth from lists to the sketch and talking to his peers. He seems puzzled. He asks Gloria to get the first sketch they made of the cabinet. She retrieves it from her backpack and unfolds it. They study it for a moment. He says something to Jesus, then takes a tape and measures—as if to confirm—the length of the cabinet. Sixty-eight inches.

Felipe continues this way, double-checking, trying to verify, looking up occasionally to snag the teacher, Mr. Devries, who, however, is helping a group across the room. The source of this vexation is a discrepancy that emerged as Felipe, Jesus, and Gloria were listing final numbers: The length of the sheet of plywood for the bottom of the cabinet—this is found on the list of materials—is sixty-eight inches. But the length of the top panel—listed on another sheet—is sixty-seven inches. This makes no sense. As Felipe explains it, exhibiting a nice shift from numbers to their structural meaning, the top can’t be shorter than the bottom, or the cabinet will look like this: and here he makes an abbreviated triangle in the air with his hands. What’s going on?

Finally, Mr. Devries is free, Felipe goes to get him, and they confer. The sketch Felipe has is inadequate, is not detailed enough to reveal that the top panel rests inside notches cut into the top of the side panels. These are called rabbet cuts. Felipe’s discomfort resolves quickly into understanding. The bottom panel extends to the very ends of the side panels, but the top will be shorter by a half inch on each side, the dimensions of the rabbet cuts. Thus the mystery of the sixty-eight inch bottom and the sixty-seven-inch top.

The depictions of the cabinet in Felipe’s plans do not provide enough information—through graphics or numbers—to enable him to figure out the discrepancy in measurement between top and bottom. Yet he must rely on these lists and sketches, for he does not yet know enough about cabinets to enable him to solve the problem readily…or not to assume that the discrepancy is a problem in the first place.

Fast forward now to the next cabinet Felipe builds, a few months after the completion of the one we just witnessed. This time there is no confusion about the length of the top and the bottom panels; that earlier episode taught Felipe a lot. And there is evidence of his emerging “cabinet sense.” This new cabinet requires a plastic laminate over its surface. Felipe is laying the cabinet’s face frame over a long sheet of plastic and tracing the outline of the frame onto it. This will give him the covering for the frame but leave two fairly large door-sized squares of the plastic. Felipe stops, takes a step back, looks the cabinet over, and then reaches for his list of measurements and a tape measure. I ask him what he’s doing.

We’re short on laminate, he explains, and here you’ll have these two excess pieces of it cut away from the frame. We’ll need to use them. But, he realizes, they won’t cover the doors themselves, because the doors will be larger than the opening; they’ll attach onto and over the face frame. So, he’s trying to think ahead and picture where the as yet uncut surplus might go. What other, smaller pieces of the cabinet could be covered. That’s what he’s about to check. When I describe this event to the teacher, Mr. Devries, he smiles and says, “That’s how a cabinet-maker thinks.”

* * *

Several times during the construction of the wall cabinet with the puzzling sixty-seven inch top, Felipe would comment on the mathematics involved in cabinet assembly. And I asked him about it myself. His comments were a bit contradictory, and the contradiction resonated with something that was intriguing me as well. At times he would note that the math is “simple,” “just numbers,” “only fractions.” At other times, though, even within the same few sentences, his face registering perplexity, he would observe that “a lot of math is involved” and that “it’s difficult.”

Felipe has taken algebra and is currently enrolled in college math; he knows what more advanced mathematics looks like. On the face of it, the math involved in cabinet assembly is pretty simple: reading a ruler; adding and subtracting (and, less frequently, multiplying and dividing) whole numbers, mixed numbers, and fractions; working with the basic properties of squares and rectangles. Yet, he says, "there’s so many pieces you need to take into consideration, otherwise, you’ll mess up somewhere.”

Felipe’s puzzlement, I think, is located in the intersection of traditional mathematics, learned most often in school, and the mathematics developed in the carpenter’s shop.

Traditional mathematics is in evidence throughout Mr. Devries’ workshop: from the calculations students do to determine cost per board foot to measurements scribbled on scraps of paper spread across the room. Considered from the perspective of school math—that is, if lifted from context and presented as problems in a textbook—the operations here would be, as Felipe observes, fairly rudimentary, grade-school arithmetic.

But as these measures and calculations play out in assembly—particularly an assembly that is unfamiliar—things get more complex, and thus Felipe and his crew move slowly and with some uncertainty. With an incomplete sense of a cabinet’s structure, Felipe must keep a number of variables in mind, arrayed in three-dimensional space, with each variable having consequences for the other. The top of the cabinet will be shorter in length by the sum of the two rabbet cuts in the side panels—but what about the width of the top? Will it rest in a cut in the back panel, and if so, what are the implications for the measurements of the back panel? Will the top extend into or onto the face frame? What does that mean for the face frame? And so on. In neurologist Frank Wilson's phrasing, this young carpenter is developing the ability to "spatialize" mathematics—and as Felipe notes, that means taking "so many pieces…into consideration." Mr. Devries tells me that he has students taking calculus who have a hard time with such tasks.

There is a small but growing research literature on mathematics in the workplace—from the tailor’s shop to the design studio—and a few of these studies focus on carpentry. Listening to Felipe puzzle over the nature of the mathematics of assembly led me to look more closely at the math in Mr. Devries’ shop, and what I saw matched earlier studies, some of which were conducted in other cultures, such as in South Africa, suggesting some cognitive commonality to the way carpenters do the work they do.

One of the findings of this research is that a wide range of mathematical concepts and operations are embodied in carpentry’s artifacts and routines, and in ways suited to the properties of materials and the demands of production. The carpenter’s math is tangible and efficient.

Take, for example, measurement. The ruler and framing square provide measurements, but so do objects created in the shop: one piece of wood, precisely cut, can function as the measure for another. Tools are also used as measuring devices. A sixteen-inch claw hammer laid sideways on a wall provides a quick measure for the location of studs in a wall frame. And carpenters use their hands and fingers to measure and compare. (“I use my forefinger and thumb for calipers,” reports master woodworker Sam Maloof.) They develop an eye for length and dimension and for relations and correspondences.

Working in the shop, the young carpenter learns a range of other mathematical concepts: symmetry, proportion, congruence, the properties of angles. Planing straight the edge of a board, cutting angles on the miter box, laying out the pieces of a cabinet’s face frame to check for an even fit—through these activities, Mr. Devries’ students see mathematical ideas manifested, and feel them, too, gaining a sense of trueness and error. Fractions were never more real to Felipe than during the episode with that cabinet top.

Portraits of Thinking: A Test Taker

I want to continue the discussion of cognition, and to do so through a series of portraits drawn from the writing I’ve done over the years.

As I’ve been arguing during the year of this blog’s existence—and for some time before—we tend to think too narrowly about intelligence, and that narrow thinking has affected the way we judge each other, organize work, and define ability and achievement in school. We miss so much.

I hope that the portraits I offer over the next few months illustrate the majesty and surprise of intelligence, its varied manifestations, its subtlety and nuance. The play of mind around us. And, though not all the portraits will be of young people in school, I hope, as well, that collectively the portraits help us think in a richer way about teaching, learning, achievement, and the purpose of education—a richer way than that found in our current national policy and political discourse about school.

This first portrait comes from an adult literacy and developmental education program that I describe in Lives on the Boundary. The focus is on standardized testing, a close look at one test-taker. And though this woman is in her forties, I think there’s a lot here worth considering for all ages, especially in our current test-intensive culture.

***

When they entered the program, Ruby and Alice and Sally and all the rest were given several tests, one of which was a traditional reading inventory. The test had a section on comprehension—relatively brief passages followed by multiple-choice questions—and a series of sections that tested particular reading skills: vocabulary, syllabication, phonics, prefixes and roots. The level of the instrument was pretty sophisticated, and the skills it tested are the kind you develop in school: answering multiple-choice questions, working out syllable breaks, knowing Greek and Latin roots, all that.

What was interesting about this group of test takers was that—though a few were barely literate—many could read and write well enough to get along, and, in some cases, to help those in their communities who were less skilled. They could read, with fair comprehension, simple news articles, could pay bills, follow up on sales and coupons, deal with school forms for their kids, and help illiterate neighbors in their interactions with the government. Their skills were pretty low-level and limited profoundly the kinds of things they could read or write, but they lived and functioned amid print.

The sad thing is that we have few tests of such naturally occurring competence. The typical test focuses on components of reading ability tested in isolation (phonetic discrimination, for example) or on those skills that are school-oriented, like reading a passage on an unfamiliar topic unrelated to immediate needs: the mating habits of the dolphin, the Mayan pyramids. Students then answer questions on these sorts of passages by choosing one of four or five possible answers, some of which may be purposely misleading.

To nobody’s surprise, Ruby and her classmates performed miserably. The tasks of the classroom were as unfamiliar as could be. There is a good deal of criticism of these sorts of reading tests, but one thing that is clear is that they reveal how well people can perform certain kinds of school activities. The activities themselves may be of questionable value, but they are interwoven with instruction and assessment, and entrance to many jobs is determined by them. Because of their centrality, then, I wanted to get some sense of how the students went about taking the tests. What happened as they tried to meet the test’s demands? How was it that they failed?

My method was simple. I chose four students had each of them take sections of the test again, asking them questions as they did so, encouraging them to talk as they tried to figure out an item.

The first thing that emerged was the complete foreignness of the task. A sample item in the prefixes and roots section (called Word Parts) presented the word “unhappy,” and asked the test-taker to select one of four other words “which gives the meaning of the underlined part of the first word.” The choices were very, glad, sad, not. Though the teacher giving the test had read through the instructions with the class, many still could not understand, and if they chose an answer at all, most likely chose sad, a synonym for the whole word unhappy.

Nowhere in their daily reading are these students required to focus on parts of words in this way. The multiple-choice format is also unfamiliar—it is not part of the day-to-day literacy—so the task as well as the format is new, odd.

I explained the directions again—read them slowly, emphasized the same item—but still, three of the four students continued to fall into the test maker’s trap of choosing synonyms for the target word rather than zeroing in on the part of the word in question. Such behavior is common among those who fail in our schools, and it has led some commentators to posit the students like these are cognitively and linguistically deficient in some fundamental way: they process language differently, or reason differently from those who succeed in school, or the dialect they speak in some basic way interferes with their processing of Standard Written English.

Certainly in such a group—because of malnourishment, trauma, poor health care, environmental toxins—you’ll find people with neurolinguistic problems or with medical difficulties that can affect perception and concentration. And this group—ranging in age from nineteen to the mid-fifties—has a wide array of medical complications: diabetes, head injury, hypertension, asthma, retinal deterioration, and the unusual sleep disorder called narcolepsy. It would be naïve to deny the effect of all this on reading and writing.

But as you sit alongside these students and listen to them work through a task, it is not damage that most strikes you. Even when they’re misunderstanding the test and selecting wrong answers, their reasoning is not distorted and pathological. Here is Millie, whose test scores placed her close to the class average—and average here would be very low just about anywhere else.

Millie is given the word ““kilometer” and the following list of possible answers:

a. thousand
b. hundred
c. distance
d. speed

She responds to the whole word—kilometer—partially because she still does not understand how the test works, but also, I think, because the word is familiar to her. She offers speed as the correct answer because: “I see it on the signs when I be drivin’.” She starts to say something else, but stops abruptly. “Whoa, it don’t have to be ‘speed’—it could be ‘distance.’”

“It could be ‘distance,’ couldn’t it?” I say.

“Yes, it could be one or the other.”

“Okay.”

“And then again,” she says reflectively, “it could be a number.”

Millie tapped her knowledge of the world—she had seen kilometer on road signs—to offer a quick response: speed. But she saw just as quickly that her knowledge could logically support another answer (distance), and, a few moments later, saw that what she knew could also support a third answer, one related to number. What she lacked was specific knowledge of the Greek prefix kilo, but she wasn’t short on reasoning ability. In fact, reading tests like the one Millie took are constructed in such a way as to trick you into relying on commonsense reasoning and world knowledge—and thereby choosing a wrong answer. Take, for example, this item:

Cardiogram
a. heart
b. abnormal
c. distance
d. record

Millie, and many others in the class, chose heart. To sidestep that answer, you need to know something about the use of gram in other words (versus its use as a metric weight), but you need to know, as well, how these tests work.

After Millie completed five or six items, I have her go back over them, talking through her answers with her. One item that had originally given her trouble was “extraordinary”: a) “beyond”; b) “acute”; c) “regular”; d) “imagined.” She had been a little rattled when answering this one. While reading the four possible answers, she stumbled on “imagined”: “I…im…”; then, tentatively, “imaged”; a pause again, then “imagine,” and, quickly, “I don’t know that word.”

I pronounce it.

She looks up at me, a little disgusted, “I said it, didn’t I?”

“You did say it.”

“I was scared of it.”

Her first time through, Millie had chosen regular, the wrong answer—apparently locking onto ordinary rather than the underlined prefix extra—doing just the opposite of what she was supposed to do. It was telling, I thought, that Millie and two or three others talked about words scaring them.

When we come back to “extraordinary” during our review, I decide on a strategy. “Let’s try something,” I say. “These tests are set up to trick you, so let’s try a trick ourselves.” I take a pencil and do something the publishers of the test tell you not to do: I mark up the test booklet. I slowly began to circle the prefix extra, saying, “This is the part of the word we’re concerned with, right?” As soon as I finish she smiles and says “beyond,” the right answer.

“Did you see what happened there?  As soon as I circled the part of the word, you saw what it meant.”

“I see it,” she says. “I don’t be thinking about what I’m doing.”

I tell her to try what I did, to circle the part of the word in question, to remember that trick, for with tests like this, we need a set of tricks of our own.

“You saw it yourself,” I say.

“Sure I did. It was right there in front of me—‘cause the rest of them don’t even go with ‘extra.’”

I am conducting this interview with Millie in between her classes, and our time is running out. I explain that we’ll pick this up again, and I turn away, checking the wall clock, reaching to turn off the tape recorder. Millie is still looking at the test booklet.

“What is this word right here?” she asks. She had gone ahead to the other, more difficult, page of the booklet and was pointing to “egocentric.”

“Let’s circle it,” I say. “What’s the word? Say it.”

“Ego.”

“What’s that mean?”

“Ego. Oh my.” She scans the four options—self, head, mind, kind—and says “self.”

“Excellent!”

“You know, when I said ‘ego,’ I tried to put it in a sentence: ‘My ego,’ I say. That’s me.”

I ask her if she wants to look at one more. She goes back to “cardiogram,” which she gets right this time. Then to “thermometer,” which she also gets right. And “bifocal,” which she gets right without using her pencil to mark the prefix. 

Once Millie saw and understood what the test required of her, she could rely on her world knowledge to help her reason out some answers. Cognitive psychologists talk about task representation, the way a particular problem is depicted or reproduced in the mind. Something shifted in Mille’s concept of her task, and it had a powerful effect on her performance.

Education, the Media, and the Public Sphere

I am so pleased to see the posts on this issue of the purpose of schooling. I’m continuing to think and write about the issue myself, and I hope this blog becomes a place for lots of people to exchange ideas about the reasons we educate in a democracy. Let me also invite readers who teach in postsecondary settings, job training programs, adult literacy centers, etc. to join in. The discussion of the goals of education has become narrow in those settings as well.

I think there is a two-fold task before us.

The first part, as everyone has been saying, is to offer a richer, fuller list of reasons as to why we educate in a democracy. I think we already have a long list of good reasons, and many of them have been posted by the readers of this blog alone. True, much public discussion and deliberation would be needed to refine them and achieve consensus, but we’re not operating in a conceptual vacuum.

The big challenge right now as I see it is to get this discussion more fully into the policy arena and public sphere. That is the second part of the task that faces us, and I think it is formidable. Let me tell you a brief story.

I originally submitted the essay on NCLB that I referred to in my 3/19/08 post to one of the on-line politics and culture magazines, a place where I had published before. You can read excerpts on the 3/19/08 post or retrieve the whole articles from the “news” section of my web site, but, in a nutshell, I tried to give NCLB its due while explaining its limitations and unintended consequences. So, for example, I explained in non-technical language why a standardized test score is an inadequate measure of learning. This is the kind of thing, I figured, it would be good for the general public to consider.

The editor responded that the piece was too “wonky” and “cautious”. Could I send something that is “faster” and gets to “what works and what should be changed?” I understand the editor’s desire for writing that has some snap to it, but I was also struck by the characterization of the writing as “wonky”, which, to my mind, means overly laden with details that only a policy buff would appreciate. How did we get to the place, I wondered, where analysis equals tedious attention to detail, where reflection becomes “caution”, or, the subtext, boring?

As we continue to try to change the public conversation about education, we need to consider, as well, the limits of the mechanisms of public discourse. I’ve been thinking a lot about this lately, and will have more to say in future posts. But I’m curious about your thoughts and experiences. How do we get more deliberative discussion about the purpose of schooling out into the public sphere?