Portraits of Thinking: A Test Taker

I want to continue the discussion of cognition, and to do so through a series of portraits drawn from the writing I’ve done over the years.

As I’ve been arguing during the year of this blog’s existence—and for some time before—we tend to think too narrowly about intelligence, and that narrow thinking has affected the way we judge each other, organize work, and define ability and achievement in school. We miss so much.

I hope that the portraits I offer over the next few months illustrate the majesty and surprise of intelligence, its varied manifestations, its subtlety and nuance. The play of mind around us. And, though not all the portraits will be of young people in school, I hope, as well, that collectively the portraits help us think in a richer way about teaching, learning, achievement, and the purpose of education—a richer way than that found in our current national policy and political discourse about school.

This first portrait comes from an adult literacy and developmental education program that I describe in Lives on the Boundary. The focus is on standardized testing, a close look at one test-taker. And though this woman is in her forties, I think there’s a lot here worth considering for all ages, especially in our current test-intensive culture.

***

When they entered the program, Ruby and Alice and Sally and all the rest were given several tests, one of which was a traditional reading inventory. The test had a section on comprehension—relatively brief passages followed by multiple-choice questions—and a series of sections that tested particular reading skills: vocabulary, syllabication, phonics, prefixes and roots. The level of the instrument was pretty sophisticated, and the skills it tested are the kind you develop in school: answering multiple-choice questions, working out syllable breaks, knowing Greek and Latin roots, all that.

What was interesting about this group of test takers was that—though a few were barely literate—many could read and write well enough to get along, and, in some cases, to help those in their communities who were less skilled. They could read, with fair comprehension, simple news articles, could pay bills, follow up on sales and coupons, deal with school forms for their kids, and help illiterate neighbors in their interactions with the government. Their skills were pretty low-level and limited profoundly the kinds of things they could read or write, but they lived and functioned amid print.

The sad thing is that we have few tests of such naturally occurring competence. The typical test focuses on components of reading ability tested in isolation (phonetic discrimination, for example) or on those skills that are school-oriented, like reading a passage on an unfamiliar topic unrelated to immediate needs: the mating habits of the dolphin, the Mayan pyramids. Students then answer questions on these sorts of passages by choosing one of four or five possible answers, some of which may be purposely misleading.

To nobody’s surprise, Ruby and her classmates performed miserably. The tasks of the classroom were as unfamiliar as could be. There is a good deal of criticism of these sorts of reading tests, but one thing that is clear is that they reveal how well people can perform certain kinds of school activities. The activities themselves may be of questionable value, but they are interwoven with instruction and assessment, and entrance to many jobs is determined by them. Because of their centrality, then, I wanted to get some sense of how the students went about taking the tests. What happened as they tried to meet the test’s demands? How was it that they failed?

My method was simple. I chose four students had each of them take sections of the test again, asking them questions as they did so, encouraging them to talk as they tried to figure out an item.

The first thing that emerged was the complete foreignness of the task. A sample item in the prefixes and roots section (called Word Parts) presented the word “unhappy,” and asked the test-taker to select one of four other words “which gives the meaning of the underlined part of the first word.” The choices were very, glad, sad, not. Though the teacher giving the test had read through the instructions with the class, many still could not understand, and if they chose an answer at all, most likely chose sad, a synonym for the whole word unhappy.

Nowhere in their daily reading are these students required to focus on parts of words in this way. The multiple-choice format is also unfamiliar—it is not part of the day-to-day literacy—so the task as well as the format is new, odd.

I explained the directions again—read them slowly, emphasized the same item—but still, three of the four students continued to fall into the test maker’s trap of choosing synonyms for the target word rather than zeroing in on the part of the word in question. Such behavior is common among those who fail in our schools, and it has led some commentators to posit the students like these are cognitively and linguistically deficient in some fundamental way: they process language differently, or reason differently from those who succeed in school, or the dialect they speak in some basic way interferes with their processing of Standard Written English.

Certainly in such a group—because of malnourishment, trauma, poor health care, environmental toxins—you’ll find people with neurolinguistic problems or with medical difficulties that can affect perception and concentration. And this group—ranging in age from nineteen to the mid-fifties—has a wide array of medical complications: diabetes, head injury, hypertension, asthma, retinal deterioration, and the unusual sleep disorder called narcolepsy. It would be naïve to deny the effect of all this on reading and writing.

But as you sit alongside these students and listen to them work through a task, it is not damage that most strikes you. Even when they’re misunderstanding the test and selecting wrong answers, their reasoning is not distorted and pathological. Here is Millie, whose test scores placed her close to the class average—and average here would be very low just about anywhere else.

Millie is given the word ““kilometer” and the following list of possible answers:

a. thousand
b. hundred
c. distance
d. speed

She responds to the whole word—kilometer—partially because she still does not understand how the test works, but also, I think, because the word is familiar to her. She offers speed as the correct answer because: “I see it on the signs when I be drivin’.” She starts to say something else, but stops abruptly. “Whoa, it don’t have to be ‘speed’—it could be ‘distance.’”

“It could be ‘distance,’ couldn’t it?” I say.

“Yes, it could be one or the other.”

“Okay.”

“And then again,” she says reflectively, “it could be a number.”

Millie tapped her knowledge of the world—she had seen kilometer on road signs—to offer a quick response: speed. But she saw just as quickly that her knowledge could logically support another answer (distance), and, a few moments later, saw that what she knew could also support a third answer, one related to number. What she lacked was specific knowledge of the Greek prefix kilo, but she wasn’t short on reasoning ability. In fact, reading tests like the one Millie took are constructed in such a way as to trick you into relying on commonsense reasoning and world knowledge—and thereby choosing a wrong answer. Take, for example, this item:

Cardiogram
a. heart
b. abnormal
c. distance
d. record

Millie, and many others in the class, chose heart. To sidestep that answer, you need to know something about the use of gram in other words (versus its use as a metric weight), but you need to know, as well, how these tests work.

After Millie completed five or six items, I have her go back over them, talking through her answers with her. One item that had originally given her trouble was “extraordinary”: a) “beyond”; b) “acute”; c) “regular”; d) “imagined.” She had been a little rattled when answering this one. While reading the four possible answers, she stumbled on “imagined”: “I…im…”; then, tentatively, “imaged”; a pause again, then “imagine,” and, quickly, “I don’t know that word.”

I pronounce it.

She looks up at me, a little disgusted, “I said it, didn’t I?”

“You did say it.”

“I was scared of it.”

Her first time through, Millie had chosen regular, the wrong answer—apparently locking onto ordinary rather than the underlined prefix extra—doing just the opposite of what she was supposed to do. It was telling, I thought, that Millie and two or three others talked about words scaring them.

When we come back to “extraordinary” during our review, I decide on a strategy. “Let’s try something,” I say. “These tests are set up to trick you, so let’s try a trick ourselves.” I take a pencil and do something the publishers of the test tell you not to do: I mark up the test booklet. I slowly began to circle the prefix extra, saying, “This is the part of the word we’re concerned with, right?” As soon as I finish she smiles and says “beyond,” the right answer.

“Did you see what happened there?  As soon as I circled the part of the word, you saw what it meant.”

“I see it,” she says. “I don’t be thinking about what I’m doing.”

I tell her to try what I did, to circle the part of the word in question, to remember that trick, for with tests like this, we need a set of tricks of our own.

“You saw it yourself,” I say.

“Sure I did. It was right there in front of me—‘cause the rest of them don’t even go with ‘extra.’”

I am conducting this interview with Millie in between her classes, and our time is running out. I explain that we’ll pick this up again, and I turn away, checking the wall clock, reaching to turn off the tape recorder. Millie is still looking at the test booklet.

“What is this word right here?” she asks. She had gone ahead to the other, more difficult, page of the booklet and was pointing to “egocentric.”

“Let’s circle it,” I say. “What’s the word? Say it.”

“Ego.”

“What’s that mean?”

“Ego. Oh my.” She scans the four options—self, head, mind, kind—and says “self.”

“Excellent!”

“You know, when I said ‘ego,’ I tried to put it in a sentence: ‘My ego,’ I say. That’s me.”

I ask her if she wants to look at one more. She goes back to “cardiogram,” which she gets right this time. Then to “thermometer,” which she also gets right. And “bifocal,” which she gets right without using her pencil to mark the prefix. 

Once Millie saw and understood what the test required of her, she could rely on her world knowledge to help her reason out some answers. Cognitive psychologists talk about task representation, the way a particular problem is depicted or reproduced in the mind. Something shifted in Mille’s concept of her task, and it had a powerful effect on her performance.

Cognition and Diversity

Last summer the Conference on College Composition and Communication (which is part of the National Council of Teachers of English) asked me to contribute to their blog on diversity. What I wrote for CCCC resonates with issues that have been emerging on this blog, so I’m reprinting, with some editing, my post below. As its name suggests, CCCC’s primary focus is on college-level writing instruction, but I think the themes in “Cognition and Diversity” have broader application. See what you think.

***

The issue I want to discuss is intelligence and the broader construct of cognition: attention and perception, conceptualizing, thinking, problem-solving, etc. We tend not to think about this cluster of topics when discussing diversity – unless we’re discussing exceptional children – but beliefs about intelligence are woven throughout beliefs about race, gender, social class, and ability.

I’ll begin with a little personal history.

I’ve been interested in the way we think for a long time. When I was an English major, I found myself drawn to accounts of a writer’s creative process: What was the inspiration for a story or a key defining moment or image that was the germ of the thing? Or what happened to a poem through various revisions; what did we know about why changes were made? Or I was fascinated by those bursts of creativity that seemed to come out nowhere: for example, how you couldn’t have predicted the intricacies of Moby Dick from Melville’s earlier novels.

Then came psychology and reading in perception and cognition, in child development, in cross-cultural studies. All this got me on the road, provided bodies of knowledge and ways to understand and study.

But not without complication.

The history of psychological and social science – and the humanities as well – is laden with research and writing that reflects the biases of the larger culture from which in emerges. So, as in the larger culture, you have claims about the intellectual inferiority of non-white races, or immigrants, or rural folk, or women. You have claims about linguistic inferiority. You have all sorts of claims about the working-class and the work they do.

I won’t weigh the present essay down with the details of how I found my way through all this and begin using the cognitive perspective toward what I hope are egalitarian ends. (Anyone interested in more of that detail can find it in a book of mine titled An Open Language.) But I do want to zero in on two things that I think are central to my own development, and are pertinent to the upcoming discussion.

One is my own background as the child of immigrant working-class parents growing up in a poor neighborhood. I know intimately many of the kinds of people who are the focus of claims about intellectual and linguistic inferiority. And what I heard and read didn’t always match up.

The second is that I started tutoring and teaching at a relatively young age in schools and programs that served poor and working-class people from a range of racial and ethnic backgrounds – and the settings spanned kindergarten to college. So, again, I saw first-hand the processes of teaching and learning, and I saw what people can do with their minds.

Both of these elements of my personal history certainly contributed to the way I saw myself, my values and dreams, and they contributed as well to an empirical and skeptical bent, useful both to question the ugliness I’d hear on the streets, on the radio, in my own neighborhood and extended family as well as the claims made in some of the academic material I was encountering.

This empirical skepticism, this need to test what I was studying against my own personal and professional experience, enabled me to use cognitively-oriented research to both critique work within the cognitive tradition that diminished human ability as well as critique the many and ongoing claims that rise like crabgrass in our society about the intellectual capabilities of underprepared students, poor folks, people of color, women, manual workers, you name it.

So let me fast-forward now to a few quick summaries of this work.

My study of cognition combined with other areas of study in the humanities and social science led to a series of articles that, collectively, tried to do the following. I wanted to explore the way flawed assumptions about cognition and language have influenced remedial writing curricula; the limiting institutional definitions of remediation and of writing instruction; overgeneralizing explanations as to why some students have difficulty with writing; and the classroom processes by which some students get defined as intellectually and linguistically deficient.

In addition to critique, I advocated a richer, more multifaceted model of cognition and writing and a way to think about curriculum and instruction that honored that richness. (Some of my earlier entries summarize this approach, see 07/08/08 and 07/24/08).

All of this work played itself out in a series of articles that you’ll find in that Open Language collection and, in more narrative form, in Lives on the Boundary.

I can give you a flavor for this writing by doing a pretty unblogospheric thing here and quoting the closing paragraph from one of the articles, “Narrowing the Mind and Page: Remedial Writers and Cognitive Reductionism”:

If I could compress this essay’s investigation down to a single conceptual touchstone, it would be this: Human cognition – even at its most stymied, bungled moments – is rich and varied. It is against this assumption that we should test our theories and research methods and classroom assessments. Do our practices work against classification that encourages single, monolithic explanations of cognitive activity? Do they honor the complexity of interpretive efforts even when those efforts fall short of some desired goal? Do they foster investigation of interaction and protean manifestation rather than investigation of absence? Do they urge reflection on the cultural biases that might be shaping them? We must be vigilant that the systems of intellect we develop or adapt do not ground our students’ difficulties in sweeping, essentially one-dimensional perceptual, neurophysiological, psychological, or linguistic processes, systems that drive broad cognitive wedges between those who do well in our schools and those who don’t.

Though some of this work is of its time (it was written in the 1980s), it unfortunately is pertinent today. Consider the number of basic/remedial/preparatory writing courses that are still built on problematic notions of cognition and language, leading to deadening skills and drills curricula. Or an article that appeared in the June Atlantic Monthly (that we’ve discussed on this blog) in which a disgruntled community college professor depicts his students as academically dense and marginally literate. Or that old bad penny Charles Murray of Bell Curve fame peddling again in True Education methodologically flawed notions about intelligence and the social order.

O.K., one more fast-forward, this one to The Mind at Work, a recent book in which I continue exploring questions of cognition, intelligence, and achievement. I blend case histories of blue collar and service workers with cognitive and social analysis to challenge longstanding Western distinctions between mental and physical activity, offering, I hope, a more psychologically and educationally productive way to consider what we do with hand and brain.

From Classical Greece on down, we have tended to make sharp and value-laden separations between the mental and physical, between the philosophical, theoretical, and conceptual versus the practical, applied, and concrete – and, more recently, between the academic and the vocational. These distinctions have affected the way we define intelligence, create curriculum, and organize work. This kind of binary thinking is inadequate to describe what actually occurs as waitresses or welders (or, for that matter, teachers or surgeons) apply knowledge, solve problems, arrive at decisions, and make aesthetic judgments.

This set of issues seems especially important for those of us who teach students from working-class families and/or who work in programs aimed at providing occupational training.

I think these issues are also important for all of us, for with our educations can come a predisposition to elevate the intellectual content and value of one kind of work over another and make cognitive judgments about people based on the work they do.

Having said that, I feel the need to explain further, and, if you’ll indulge me one more time, I’m going to do the boorish thing of quoting myself again, this time from The Mind at Work:

This is not a call for a simplified egalitarianism. I am not denying the obvious fact that people come to any pursuit with different interests, talents, knacks for things, motivations, capabilities. Nor am I claiming that all bodies of knowledge and expressions of mind are of the same level of cognitive complexity and social importance. All the cultures I’m familiar with make judgments about competence in the domains that matter to them. (Though ours is more obsessed than any I know with developing measures of the mind and schemes to rank them.) No, the distressing thing is that both in our institutional systems and in our informal talk we tend to label entire categories of work and the people associated with them in ways that generalize, erase cognitive variability, and diminish whole traditions of human activity. Attributions of merit and worth flow throughout the process. We order, we rank, we place at steps upon a ladder rather than appreciating an abundant and varied cognitive terrain.

In closing, let me offer a cautionary tale that illustrates how easily overgeneralized and ungenerous judgments about other people’s thinking can come to us.

About ten years ago a graduate student came to see me with a sketch of a dissertation proposal. It had taken this person a fairly long time to get to this place, having begun then abandoned a previous research topic. And the sketch I was looking at was also the result of many months of deliberation. Along the way another faculty member had commented to me that this person was a “weak” student.

I read the new sketch, and it wasn’t good at all. It was general in some places. In others, one claim didn’t line up with the next. Some sentences were difficult to understand. It was hard to know exactly what the research project was. The comment from my colleague flipped into my mind like a pop-up ad. And so did a sense that’s hard to describe, but was kind of a half-thought/half-feeling that this student might not have the ability to complete a dissertation.

We met and caught up a little, stuff about family and work. Then we turned to the proposal. I decided to avoid its problems and asked the student to talk to me about the project, not in dissertation lingo, but in everyday speech.

What followed was clear, elaborated, interesting. A solid, engaging project. We talked a while longer, getting some notes down on paper. I then turned to the piece I’d read and pointed out a few places where I had had trouble. And the student explained – frustration seeping out – that what I read was an attempt to reconcile conflicting advice from another faculty member, several peers, and an activist in the community to be studied.

This student’s dilemma is familiar to all of us, I’m sure – the way conceptual (and interpersonal) conflicts can negatively affect our writing. But look what went on in my head when I first read the proposal sketch. Without realizing it, I had absorbed the informal norms of graduate study: that, for example, time-to-degree is a measure of ability or that flawed writing equaled flawed thinking. I had drunk the cognitive Kool Aid.

As I write in that paragraph from The Mind at Work, I’m not trying to ignore the fact that we, all of us, do have different talents, interests, etc. It is possible that the student was, for all sorts of reasons, not ready or equipped to write a dissertation. And, after all, as educators we’re obliged to make judgments about performance and respond accordingly. What is troubling in the anecdote, however, is the ease with which a one-dimensional judgment about intellectual ability came to me.

But the anecdote also points to some ways out of this mess. It reminds us that we live tangled in systems of bias, and that we will always blunder, and, therefore, we need in our teaching some methods to keep us aware, some tools of mindfulness: asking different questions, shifting languages, listening closely. We need certain habits of mind, for example, a testing of our own judgments, a willingness to have them disconfirmed. We need to be alert to the social contexts we inhabit – this was the root of my error – and the norms and beliefs we absorb in them. We need to publicly question the vocabulary and assumptions that constitute these settings. (This blog is a tiny gesture in that direction.) We also need to be creative in fashioning other kinds of spaces within those worlds we inhabit.

These are the kinds of issues and questions we – I – need to keep raising. They keep in sight the ease with which we reduce each other. They contribute to a richer pedagogical imagination. Ant they can help fashion a more humane institutional and civic life.