Over this last weekend, the gigantic inequalities in Virginia’s NCLB-waiver plan became a target of criticism by former Virginia state board member Andrew Rotherham, among others. Essentially, Virginia decided to set different academic achievement expectations by race and ethnicity, and many have legitimately pointed out that such policies allow many schools to evade responsibility for inequality. The same thing happened in New York City in the early 1970s when the public school administration was ready to contract with a testing firm with different expectations by race. In that earlier case, famous psychologist Kenneth Clark righteously criticized the NYC schools for its unequal standards. ((I describe the New York City events early in Accountability Frankenstein.))
There is no easy algorithmic solution, either by setting identical standards by achievement status (the NCLB method of 100% proficiency), by some measure of growth (the current fad), or by some hybrid (such as NCLB’s rocky Safe Harbor provision). As Jennifer Hochschild and Nathan Scovronick explained, accountability pressures focused on status can lead to unreasonable demands on teachers and students, but plenty of other approaches (either focused on some measure of growth or by setting different expectations) can lead to inappropriately low expectations. And let there be no doubt: algorithmic accountability systems are clunky at best, often fiddled with to avoid public disclosure of politically-unacceptable failure. But as long as people prefer algorithms to make political judgments, we are going to get plainly stupid algorithms more often than not. Virginia’s waiver-request expectations are the close cousins of 100% proficiency.
Fortunately, there is an out: use algorithms to screen for closer examination through something like the older British school inspections. Tests can be used to identify either schools or students who need closer attention, but the final judgment of what needs to happen in schools should be a human judgment.
I like this idea in theory, but NYC has something called the “Quality Review” that works similarly. Initially, all schools got a QR, though largely for budget reasons they later revised the approach to allow exemptions for schools that performed well on data-based accountability measures. In all cases, however, the QR proved to have limited utility because it’s very gameable. Essentially, once you create a rubric to evaluate a school, even mediocre can perform to those expectations if only for the purposes of the inspection. This is not especially different from the reality that teachers (and their students) often behave differently when they are being observed by a school administrator. Much like teacher evaluations, no individual measure is really sufficient and the best approach to evaluating schools should probably account for multiple, weighted measures rather than saying one particular measure trumps the others.