November 2003 Archives

Another criticism of standardized tests is that they cannot measure the sort of understandings we expect the schools to be teaching such as writing and scientific inquiry. As Audrey Qualls, a co-author of the Iowa Test of Basic Skills, one of the most widely used standardized tests, states, �It may be very, very important to teach students how to actually set up an experiment. That's a major goal of science. ... I can't measure that very well with a multiple-choice type test.� There are a number of important skills that a test can�t measure: �Whether or not a student actually asks questions. Whether they use background information appropriately. Whether they're part of a conversation.� She goes on to put this line of thinking better than I could, so I�ll quote her at length: When I think about what a school should do or what makes a school good or bad, it's far more than how well they teach a few sets of skills. When we think about what we've [asked] schools and teachers to do, it's the developmental growth of these little five-year-olds and six-year-olds that come in a door, all the way to being productive citizens, able to go to college or able to go on to work. And if I think about, what does that mean, it truly has to be more than just achievement in a few basic areas. One, I want them to develop an appreciation and value of societal norms. Being a good citizen. Caring about learning. Contributing to your community. I can't measure that with a test. When I think about what makes a good school, I want to think about, do teachers have the commitment to help a student who may not be the traditional student, who needs a little bit more? ... A test score doesn't capture that. ... So if I were to try to say, "OK, all of that sounds wonderful. And I want the school to do it," but all that I can easily measure is your achievement over some basic skills, I've cheated the schools. ... All quotes from interview for frontline.

The testing is the most contested part of this whole program. One of the first things to understand is the difference between the sort of tests your teachers made in high school and standardized tests. When teachers make a test, they survey the facts and skills they expect students to have and design a task that will require those. Standardized tests are designed to spread students out over a normal curve so they can be compared; the items are chosen because they are easy, medium, or hard, not because they represent all the material to be taught. So the first and most obvious criticism is that the tests aren�t actually measuring what the standards say they should. What�s worse, though, is that standardized tests are created so that no matter what, 50% of the kids who take it score below average (that�s what average means after all). But that means by definition, if we set the 50th percentile as proficient, it is impossible to have 100% of students score at the proficient level. This is such an obvious criticism, it is hard to believe a satisfactory response is not part of the policy itself. Yet California uses the SAT-9, a standardized test said to be sufficiently aligned with the standards to measure them, even though it was designed as a norm-referenced achievement test and will continue to fail half the students in the state. What�s needed, clearly, are new tests designed to verify the subject-knowledge outlined by standards, but such tests are very, very expensive, and take three to seven years to develop. NCLB set aside quite a bit of funding for developing such tests, but this year Congress approved less than two thirds the amount the bill called for.

The bill�s basic mechanism is to set standards and then test to see if schools are meeting them. Not surprisingly the two most contended parts of the policy are standards and testing, and that is where most of the research has been done.
Standards are supposed to spell out simply and clearly precisely what students whould understand and be able to do at the end of each year. However, most of the standards include too much material to be feasible and are too vague about what students should understand. William Schmidt, who studies education systems in �competing� first-world countries explains, �We're, for example, at eighth-grade telling teachers to teach 35 topics. Other countries are telling their teachers to teach 10 to 15. So there's one aspect of it. But secondly, the standards themselves are often times not very clear or focused, so that a teacher could say, �Of course, I cover that.� And in some sense, they do. But they're over on this corner of it, versus the real depth of the standard, which is over here.� For example a standard may say "Students will understand historical events in the twentieth century," leaving a teacher to wonder whether students should merely be able to list the events, or be able to offer illustrative accounts of each. In sixth-grade Social Studies, teachers are supposed to cover world history from the agricultural revolution through the Rennaissance, about seven thousand years including the incredibly dense cultural innovations of irrigation, myth, writing, democracy, literature, philosophy, the scientific method, and guns. Amy Wilkins of The Education Trust explains the way the standards were created, �in general, there were very open processes, with lots of people, mathematicians, parents, all sorts of people deciding what kids should know and be able to do. And they produced these very thick documents, and sort of threw them at teachers and said, �Here, do this,�� As a result, the standards are not as clear or focused as many had hoped.
When the standards are so impossible to cover and vague, the standardized tests used to measure them become the guide for classroom instruction. In such cases, teachers must �teach to the test,� perhaps the most prevalent problem with No Child Left Behind.

NCLB went into effect two years ago, meaning that each state should now be 1/6th of the way to having all students proficient in all subjects. Schools are starting to fail and be put on the underperforming list. In New York and Chicago, so many parents of students at underperforming schools have requested to have their children transferred to schools that are meeting the standards that the districts are swamped. There are not enough vacancies at schools performing well to accommodate all the inner-city school refugees. Both school districts now stand in violation of the federal law (as well as many others I expect) but no one has yet brought suit. In Florida, Tennessee, Missouri and West Virginia, nearly half or more of schools are not meeting the new benchmarks (source). We are beginning to see the effects of Bush�s policy and the failure of the federal government to afford the amount of time and money it will take to �leave no child behind.� Clinton had a similar program called �Goals 2000� that failed to make any difference and simply faded into dust when schools did not meet the mark in 2000. Will No Child Left Behind simply be ignored and forgotten? For all its flaws, I hope not. Many inner-city school districts have said that it�s pressure is their only hope for the reform they need.

Last night I attended my English teacher preparation class, where we were given the second of two relatively major assignments. Amongst the requirements for the paper was the criterion that we cite six scholarly sources, at least half of them from peer-refereed journals in the last year or two. The outcry was tremendous. �Why such an emphasis on theory in a practical course?� �How are we supposed to have time to teach and go to the library?� �How will that help me teach my class tomorrow?� I was stunned. If teaching is indeed a profession, like medicine or law, one of its hallmarks should be the need to keep abreast of current developments by reading professional journals. They should be reading them anyway, or at least regretting that they're not. I don�t know what to say to these teachers or how to face the public and claim teachers deserve to be treated as professionals after such a show. I am merely heartbroken.