Jeff Henig, writing last week as a guest at Rick Hess’s blog:
[T]he high promise of policy by algorithm mutates into cause for concern when data are thin, algorithms theory-bare and untested, and results tied to laws that enshrine automatic rewards and penalties.
Technocratic accountability policy happens to be one of my favorite fads to study, and Henig is writing just a week after Henry Farrell blogged the following in an entry the previous week at Crooked Timber that started a series of blog entries all over the place:
Neo-liberals tend to favor a combination of market mechanisms and technocratic solutions to solve social problems. But these kinds of solutions tend to discount politics – and in particular political collective action, which requires strong collective actors such as trade unions.
Is accountability policy the type of neoliberal mechanism that gives Farrell and others the heebie-jeebies?1
First, a bit of perspective: Farrell is right to be concerned about mindless acceptance of any and all technocratic policies, but he is wrong about the assumption of political vacuity. Paul Starr and Theda Skocpol are well-known sociologists who were writing a few decades ago about the long-term political legacies of policy structures, and in many ways the political strategy of health-care reform in the early Obama administration was shaped by those arguments as well as by the failure of Clinton’s efforts. There are plenty of Farrell-tagged neoliberals who understand the political stability of universal programs such as Social Security, and while that isn’t the type of political theory Farrell may be thinking about, and it doesn’t guarantee that all neoliberalish policies or people are politically savvy, it is not discounting political dynamics.
- Statistical accountability mechanisms fall into a category of mechanisms that allow political bodies to outsource “fact-finding” to groups or mechanisms that are declared by fiat to be disinterested. Turner used juries and “blue-ribbon” commissions as his prime examples, but the category also includes expert groups such as the National Research Council and the types of algorithms that worry Henig. The Independent Payment Advisory Board in the Affordable Care Act is just such a body, as was the Defense Base Closure and Realignment Commission. The point here is not whether such mechanisms are truly disinterested but the role they play in the social determination of what reality is.
- These mechanisms in turn are part of the modern state apparatus in defining reality, or “seeing like a state,” as Scott put it. Scott’s emphasis has been on documenting the pathology of attempts to rationalize society, but I think the more general contribution of his work is on the efforts of state data collection to define society. David Nye made a similar point about the Land Ordnance of 1785 and the process of surveying the land as a state act. To Nye, the act of surveying was an important part of the effort to remake the land in North America and both pretend that and create an ideology that white settlers in the U.S. were expanding into a virgin land.
Once you recognize this impulse, it’s easy to identify hubristic state interventions as Scott has and as Henig worries about accountability algorithms. Interventionists often delude themselves into believing that they are technical wizards, that their norms are universal, that their prejudices do not influence technical mechanisms, that inaction is worse than erroneous action, etc. And if all technocratic mechanisms were inevitably broken and irreparable, I could focus on political battles over values. Or I could be a libertarian, trade slips of silver for my omelet, and be happy.
But not all technocratic mechanisms suck. Some things work remarkably when designed well: bridges, airplanes, cars, the computer I’m typing on. And a bunch of government programs really work well: the Earned Income Tax Credit, SNAP (formerly food stamps), most single-payer health care programs (in other countries), as well as safety regulations of loads of things from building codes to the engineering marvels I listed above. Since moving to Tampa 15 years ago, my life has been saved several times over by the regulation of how well cars must survive crashes, and the same is true for my spouse and my children. So thank you, safety technocrats (if not to my fellow Tampa drivers who drove their vehicles into the back end of ours).
The obvious question is, how do you tell the difference between technocratic suck and technocratic doesn’t suck? I would rather reframe it as a question of moving from one to the other, and because the primary weaknesses of technocracy are incompetence and hubris, the best remedies are to chip away at both:
- Competence matters. Developing some capacity for competent application of skill takes time and formal training. For example, it has taken two decades to develop even a relatively small population of people with the technical skills to implement growth models. There’s a larger population of folks who can read those results with some understanding, but that is different from the craft of getting complex iterative procedures to converge with anything other than gobbledygook as a result.
- Transparency matters. Incompetence and mistakes can hide in a technocratic mechanism hidden from view. That is one of the more important reasons why overemphasizing student outcomes in teacher evaluation is so dangerous: bureaucracies will obscure the potential errors of such calculations because the timeline of producing cautious, professional statistics is jarringly at odds with the need to make judgments based on annual evaluations of teachers. In the one public case where different outfits looked at test-score growth measures in the same jurisdiction (LAUSD), the results were very different in the reanalysis.2
- Having to defend views in a competitive political arena matters. Because their mechanisms are often born in the fictitious belief of neutral fact-finding, technocrats are especially vulnerable to hubris if too-well insulated from public comment. One role of vigorous public teachers unions is precisely to fund and keep visible the criticisms of technocrats.
My general view: the way one eliminates mistakes in technocratic mechanisms is to make sure it is transparent and subject to vigorous public criticism. That type of crucible will be the best assurance that we burn all suck out out of the technocrat.
So, back from that to the general question of formulaic accountability policies and neoliberalism: do technocratic accountability policies ignore the politics of collective action? My experience and my research suggests the opposite: while I am as concerned as Jeff Henig with the dangers of accountability algorithms, the development of such formulae is deeply infused with political judgments and understandings.3 The labeling of schools with an A-F scale in Florida was political brilliance, as I have stated before.4 Until NCLB, states that developed complex accountability systems also generally tinkered with the details so that most schools could succeed within the system at some point and that failure on graduation tests remained below the level that would be politically unacceptable. In Florida, the state’s official measure of learning gains starting in 2000 was not one of the complex value-added measures researchers write about but a crude yes/no measure that reduced the logistical burden on the state department of education and its contractors. It was jerry-built in the most literal sense, being crafted under the then-head of accountability, Gerry Richardson (a genuinely nice guy from everything I know).
Given the porous nature of putatively apolitical algorithms, I think it is foolish to assume that the technocratically-inclined have some neoliberal false consciousness.5 Like Henig and also folks such as Rick Hess, I think there’s considerable hubris in the formulaic-accountability crowd. But there are cautious and smart ways to use statistics. There are even folks inside collective-action groups (such as unions) who know about statistics and algorithms. At least when looked at inside education, this is an appealing but false dichotomy.
- There is also an interesting followup from last Saturday by John Quiggin largely about public-sector capacity without taxing high wage-earners, but that is beyond the scope of this entry, as well as the question of when/where/how changes in inequality has macroeconomic consequences. [↩]
- The point for those who didn’t read the paragraph is not whether one analysis was competent and the other incompetent but that they were different…. Yes, I know that the data used were somewhat different, but for those who are curious, I’d be willing to wager a good dinner on those differences existing with the same data used. [↩]
- You would see the same thing with funding formulae; at least in the case of Florida, a legislative equalization law in 1973 was tinkered with in 1974 to address political needs. But I think I’d modify the following statement Deanna Michael and I made in our chapter on Florida finance reform from Education Reform in Florida: “The resulting funding program was politically negotiated rather than technocratic–a feature that helped it survive over the next thirty years” (p. 62). The political tinkering with the formula did not make it less technocratic. [↩]
- I disagree with the A-F scale as policy, but I can admire the political ingenuity. [↩]
- Please don’t get me started on “false consciousness” and other Althusserian nonsense. [↩]