The incentives are critical, but they’re not responsible for critical choices

Today's David Brooks op-ed column ends with a mild zinger about overzealous test-prep being the choice of individual school officials rather than inevitable from high-stakes testing systems. That is precisely the argument I made in class yesterday with a handful of students, and it is absolutely in the capacity of school officials with authority to at least partially buffer teachers and students from inappropriate distortion of instruction. But it is not inherently at the level of the school where those choices are often made, despite the claims by Brooks. In Florida, a number of school systems put fairly extraordinary efforts and funding behind test-prep, inserting an additional layer of more intrusive varieties of testing and replication of test formats into the lives of schools. So it may be the fault of the teacher or principal if your child is subject to unreasoning test-prep. But it may also be the fault of the school district, and in Florida, that is nominally in violation of the law that requires school districts to set policies prohibiting test prep from interrupting real instruction. (Never mind that I have yet to see a school district with such an explicit policy, but that was something legislators did primarily to appease parents rather than to change behavior.) 

There is a deep inconsistency in the arguments of high-stakes advocates that test-based accountability policies have absolutely no relationship to test-prep. To the advocates, test-based accountability is essential to hold teachers' and principals' feet to the fire for teaching children, and the theory of action requires that accountability create incentives to change behavior. But when it comes to complaints about test-prep, all the pressures of the tests disappear: don't look at me, it was the guy next to me whose fault it is! Or the guy below me in the bureaucracy… Fundamentally, dear high-stakes advocates, you have to decide: are high-stakes accountability policies supposed to change behavior or not? 

One should also note that Brooks chose to highlight charter schools as examples of how one can respond to accountability without test-prep. It is as if he could not find a single example of a school overseen by a single regular school district that has escaped the pressures to engage in test prep. Or that he along with other pundits have a severe availability bias in favor of charter schools. It is an unfortunate bit of hype; while some charter schools do very well compared with local public schools (as would any random sample of schools), a good portion fall near or below the recorded achievement of local public schools. Yet Brooks and others continue to hype charter schools, much to the detriment of public debate. In the real world, meanwhile, silver bullets continue to be a rare commodity: Jacksonville's KIPP school fell in the lowest rating band in Florida this year. I take any state rating system with a few pounds of salt, but there will always be a consistency problem for overhyping school reformers and their fans: test score results are fragile. Maybe the Jacksonville KIPP school didn't engage in enough test-prep?

Addendum: One person reminded me this afternoon that Paul Tough describes an incredible test-prep focus in the HCZ Promise Academy in his book about Geoffrey Canada, who kicked out both employees and students when the first generation of charter-school students scored low on tests. 

3 responses to “The incentives are critical, but they’re not responsible for critical choices”

  1. Stuart Buck

    We need a more sophisticated discussion of what “test prep” or “teaching to the test” really means, and why (or when) it is supposedly so bad.

    If the test covers things that we want kids to know, then teaching ought to be to the test, in the sense that the content taught ought to be the stuff that we want kids to know. Put another way, the test and the teaching ought to be aligned (we all recognize this in the context of a teacher’s own class — the final exam should match what was taught in the class, and vice versa). Anything else is unfair to the students.

    If the stuff that gets taught ends up being too narrow, then the real problem isn’t the very existence of tests but the narrowness of a particular test. The answer is to redesign the test.

    On the other hand, if “test prep” means not aligning the content to the test, but instruction that literally does nothing except explain how to take a test — how to guess at multiple choice questions, for example — then it could be a bad thing if taken to extremes. Even here, though, learning how to guess at multiple choice questions is a logical skill that could be more broadly useful.

    At the same time, if test prep is taken to extremes, such that practicing test-taking crowds out learning actual content, then it seems like it would be counterproductive. Even Diane Ravitch makes this argument, though without following it to the logical conclusion that counterproductive teaching is a fault of the teacher or principal, not the test.

    So as to your inconsistency point, someone could say, “Yes, tests should change the behavior of schools and teachers in the sense of forcing them to focus on the things that kids really ought to know, rather than letting individual teachers idiosyncratically decide to skip the quadratic formula or spend all semester on dinosaurs just because the kids like it. On the other hand, there’s no inherent reason that tests should make schools and teachers suddenly drop all their art and history instruction and do nothing but teach kids how to guess at multiple choice questions. When schools do that, they are acting foolishly, and it’s not the test’s fault.”

    At least that would be one response.

    1. CCPhysicist

      But what if the quadratic formula isn’t on the test? You might be surprised by what some states consider “HS math”.

      And even if it is, what if (as is likely) there is only one question that requires knowing the formula and applying it correctly? Can you imagine a teacher being told to skip that topic and focus on topics the students really need to get right to “pass”?

  2. CCPhysicist

    How does he know that “tests are not the end” in EVERY charter school, and that the are “the end” in EVERY public school? Because one group of school managers told him they don’t teach to the test, and critics of the public schools say they do?

    He doesn’t, unless he has made an unannounced visit (practically impossible in today’s security conscious schools) to a random sample of schools during the weeks before the test is given, or has kids in a large random sample of schools. Unlikely.

    But wait. He says about tests in charter schools that “They are a lever to begin the process of change.” Well, they are in public schools as well, according to the principals. And that “process of change” looks a lot like prep for a particular style of test.

    And it is hardly a surprise the New Orleans students have improved since the disaster that was life there in the first few years after Katrina hit at the start of a school year.