There is a certain piquancy in the utter solemnity with which Wikipedia entries assume the truthfulness of Baumol’s cost disease and Campbell’s law. The first is a putative truism that labor-intensive fields have lower productivity increases by dint of that trait, and the second is another putative truism that all stakes on measures distort the trustworthiness of said measures. I’ve explained before why the so-called Baumol’s cost disease is an artifact of selection from a distribution (briefly: over any period, some sectors are going to have greater measures of productivity increases than others). When I saw Sharon Nichols, Gene Glass, and David Berliner use the concept of Campbell’s law in their writing about high-stakes testing, for me it came under the category of “okay, you’re trying to explain a concept and here’s something that is close enough it’s easy to grab.”
It’s been picked up enough by others, and portrayed as some iron law of human nature, and that portrayal is neither accurate nor helpful to those of us who are critical of high-stakes testing. Of course there are unintended consequences of system high stakes’ being tied to measures, consequences that one could call a distortion of the original measure. Because this is human behavior in a systemic context, the extent of such a distortion is an open question, and to assert that there is a level of corruption that is uniform (or uniformly high) is both unrealistic and closes off all sorts of avenues for interesting and useful research. How people game systems is an important area of potential research precisely for those of us who are worried about the national culture of test prep.
Here’s why, or one example: if Campell’s law is universal with uniformly high and corrupting consequences, then our broad culture of high-stakes testing makes any test whatsoever carry the same social-psychological environment as something that really carries high stakes. Formative testing? Useless, because it’ll generally carry the patina of high stakes testing, and there goes your practical and political alternative to high-stakes testing.
I don’t really believe that, because I don’t think the corrupting effects of high stakes is universal. I just don’t know where and how it is most or least influential.
The distorting effects of testing can be understood in a broader context, as in this WSJ look at our society’s obsession with “maximum performance.” Long the focus of sporting events, artificial environments are socially-constructed to intensify emotion, and the drama of winners winning, losers losing. But, as we know with testing, off-field doesn’t necessarily correspond with on-field performance.
Measurements That Mislead by Jonah Lehrer (4/2-3/11, C12)
http://online.wsj.com/article/SB10001424052748704471904576230931647955902.html
This moves the discussion beyond Don Campbell’s observations.
I can’t tell if what I see is a corrupting influence of the exams themselves or the teaching approach dictated by the powers that be. Either could explain students who ignore details in a problem statement, basing their solution on a single noun, and assume that everything SHOULD be forgotten as soon as the latest exam has been passed.
… beyond Campbell because it points us in the direction of the social context — and the society — that creates these artificial stress-bubbles for attribution and sorting purposes, as a way to justify salaries, opportunities for advancement, and especially the allocation of resources, prestige and power. How strange!