In the United States, it’s the middle of the week after Thanksgiving. You’ve had several days of work in a row with weeks of work staring at you before the next day off. We’re heading towards winter. Grover Norquist has more appearances on television this week than you’ve ever had in your life. And Thomas Friedman just suggested that Arne Duncan be made Secretary of State. It’s time for a respite and some common sense, and not just about the diet before whatever holiday you celebrate next month?
You’re in luck.
Evaluation components in a screening system, not a single algorithm
First, you’re in luck with Doug Harris, an economist at Tulane who points out on the Shanker Institute blog that combining value-added measures of student achievement with ratings from classroom observation in a quantitative algorithm is not the best way to implement Lee Shulman’s “union of insufficiencies” and answer my question about how to use bad measures in decisionmaking. Instead, Harris proposes to see different types of evidence as ways to screen for further examination. His idea does not address all concerns with using student outcome information in evaluation, but his argument is a breath of fresh air in a now-stale policy environment. (Confirmation bias disclosure: I made a similar argument about school-level evaluation in the last chapter of Accountability Frankenstein.)
Partial credit (for accountability purposes) for GEDs and graduation after the fourth year of high school
Second, I think that there is an easy resolution to a continuing dispute over the calculation of high school graduation rates and NCLB waivers, which Ed Week’s Michele McNeil covered today. Here’s the gist of the issue as I see it: a 2008 federal regulation requires that states calculate high school graduation rates in a way that follows ninth graders through high school and that eliminates a great of game-playing that states have engaged in with so-called “dropout rates,” and that are quite different from the various estimation methods developed before the regulation was effective. The definition excludes GEDs and graduation after the fourth year of high school from the graduation rate numerator, so schools do not “get credit” for students who drop out of a regular program and enter a GED program, nor do they “get credit” in the federal graduation rate for students who graduate in a fifth year of high school. Despite this regulation, a number of state NCLB waiver plans include either GEDs or post-fourth-year diplomas in their waiver-plan graduation rate calculations. California Rep. George Miller has protested that pattern in a letter to U.S. Secretary of Education Arne Duncan, and on Monday Duncan issued a “dear colleagues” letter to state superintendents about the graduation rate calculation and NCLB waivers.
At one level, the exclusion of GEDs makes a great deal of sense: it eliminates a perverse incentive that schools in Florida and other states had in pushing students out of the regular program and into adult GED programs. And having the recently-published state-level data on graduation is enormously important. Yet, at the same time, I have concerns about entirely eliminating credit for GEDs, and I have some very deep concerns about not giving schools credit for getting students to graduate in five years. If it’s a standard academic diploma, it’s an academic diploma, and I do not want schools to have an incentive to ignore students who do not appear to be on track to graduation in four years.
There are two obvious solutions. One would be to calculate two graduation rates: the strict four-year federal graduation rate, and an expanded rate that includes GEDs and post-four-year diplomas (this measure would lag a year or two). The second solution is to have a single rate (again lagging) that would give partial credit to a school for students who earn a GED or a post-fourth-year academic diploma. For arguments’ sake, if a school received credit equivalent to one-half of a four-year academic diploma for either a GED or a fifth-year graduation, then schools would not have a clear incentive to push out students, and they would be recognized for increasing on-time graduation much more than for either late graduation or for GEDs of former students.
Now, does Nate Silver care to guess the probability of either idea being adopted?
Addendum: See the comments on Harris’s entry, especially by David Cohen and Carol Corbett Burris. More generally regarding discussion about VAM, see John Thompson’s thoughts on an earlier post by Matt Di Carlo about the use of VAM in teacher evaluation, and especially the comment thread. As Rick Hess predicted,1 overcaffeinated value-added enthusiasts have poisoned the well for discussion of value-added measures.
- Though I disagree with Rick’s claim that IMPACT was a “smart, careful effort.” It was a rushed, overcaffeinated effort at a smart, careful effort. [↩]