Late one Wednesday night in late July, I became entangled in an interesting Twitter thread about testing. It began with Jennifer Borgioli’s tweet:
https://twitter.com/DataDiva/status/360208829687926784
… which she followed up with
https://twitter.com/DataDiva/status/360210628901744640
NSVF’s Benjamin Riley approved at least of the first:
And that’s what I saw. I’ve seen that argument before, most clearly about a quarter century ago when the University of Pittsburgh’s Lauren Resnick made the case for a different type of test. That was the era of “authentic assessment” (more accurately termed performance assessment, though that’s a different debate). And it didn’t exactly work out the way Resnick wanted: we still have pretty lousy multiple-choice tests for the most part. I tried to point that out, and then Riley headed in a very different direction.
https://twitter.com/shermandorn/status/360218169572474881
Whoops — that should be Lauren Resnick, not Laura.
https://twitter.com/shermandorn/status/360218584829542400
https://twitter.com/shermandorn/status/360219559942295552
https://twitter.com/shermandorn/status/360220543531745280
https://twitter.com/shermandorn/status/360222821181440000
https://twitter.com/shermandorn/status/360225355589632002
https://twitter.com/shermandorn/status/360229303725735938
At first glance, this looks like a tangent by Riley: I’m pointing out that maybe we shouldn’t rely on either of the new-generation test consortia (PARCC or SBAC) as the sole response to concerns about test-prep, and Riley heads off into the direction of “testing is a good instructional strategy.” But I don’t think it’s that much of a tangent. I think Riley and other defenders of very-high-stakes testing really think that we don’t need to worry about test-prep because little faux tests can help children learn.
Is Riley drawing a sensible conclusion from the cognitive-psych literature?
In this argument, Riley is relying on the research on retrieval practice (such as the items Riley points to, or Daniel Willingham’s explanation of them), and I draw a different conclusion. The recall/retrieval research suggests that if you persuade people to think about what they’re supposed to be learning, they will learn better. I’m not sure the research in that area is specific enough to say whether there’s a big difference between constructing the task as an open-ended “tell me everything you remember” task or something like a multiple-choice quiz, but the bottom line appears to be attention to what you’re trying to learn.
This is not rocket science, but it’s an important conclusion, and it has different implications from what Riley draws. Riley’s argument is essentially, “Testing has a good effect on learning, so we don’t need to worry too much about test-prep.” But if the key issue is making students think about what they’re supposed to be learning, we need to balance the value of a particular activity against all the activities that are reasonable alternatives: which is most likely to engage students in thinking about what you’re hoping they’re learning? Here (and wherever they pooh-pooh concerns about test-prep), Riley and other defenders of very-high-stakes testing are implying that the little faux tests and worksheets associated with test-prep are likely to be the best use of time, the activities most likely to engage students in thinking about a subject.
If you believe that, I’d like to sell you a bridge. Specifically, the one at 40.70569°N 73.99639°W.
My skepticism doesn’t mean that everything labeled “test prep” is bad. As I told my children when they complained about test-prep in the Tampa, Florida, public schools, take the label “FCAT prep” off the top of a page and ask whether what you’re doing is good regardless of the label. (I knew at the time that many teachers were labeling ordinary good instruction as test-prep to address parent pressures about test performance.) But by the same token, you cannot hide unconcern about teaching to the test under the blanket claim that the cognitive-psych literature suggests that thinking about a subject helps you learn it, and sometimes mini-tests perform that function. ((There are other questions one should ask about the generalizability of the research Riley is referring to; in general, it frames successful learning as the ability to recall information later. That’s a narrow way of looking at learning.))
So how can you judge whether something that is being pushed at least nominally in the name of test-prep is a good use of student time? You can ask the following questions:
- Take the label “test prep” off the activity. Would you want a child in that class engaged in that activity even if there were no high-stakes test on the horizon?
- Are the students actively engaged in the activity in reality, or are they going through the motions without attention?
- Is there a better use of the time, an alternative activity that is likely to be more engaging, more likely to hold the attention of students on what they are learning?
The reality is that we don’t have a great measure of how much useless test-prep occurs in schools, but my sense is that it is much more widespread than Benjamin Riley, Jeb Bush, or many others are willing to admit. And to pretend that just because testing might be a reasonable way to keep attention on a topic is justification for test prep to the exclusion of other, better ways of drawing that focus is both wrong and the road to easy self-delusion.