Duncan’s legacy: The cold take

The Conversation has just published a short piece on Arne Duncan’s legacy by ASU doctoral student Amanda Potterton and me. Below is the full text and then some comments on the writing process and what we left out:

Arne Duncan’s legacy: growing influence of a network of private actors on public education

Sherman Dorn and Amanda U. Potterton

Arne Duncan is leaving the US Department of Education in December. Reactions to his legacy have been mixed. Some see him as a heroic reformer, and others a well-intentioned but overreaching bureaucrat. He has been called the third secretary of education for George W Bush or the center of stormy education politics.

As researchers of education policy, we see him differently: the hub of a network of policy advocates. As the head of the federal Department of Education, he actively facilitated private actors’ influence on public education policy.

Continue reading “Duncan’s legacy: The cold take”

Raising Arizona’s children — a newcomer perspective

As a newcomer to Arizona since June 2014, and with an absorbing job that has been my focus for the last 15 months, I have largely stood back and watched the state’s education politics. For a small (-population) state such as Arizona, it has been an eventful 15 months:

  • The (former) incumbent state superintendent was overthrown in the 2014 Republican primary because of racist comments he wrote on the internet.
  • The woman who beat the incumbent state superintendent in the primary won the general election in the closest statewide race of 2014, by approximately one percentage point, or 18,000 votes — which I would consider a Florida landslide.
  • The new state superintendent has been in regular political battles with the state board of education, including several trips to state court over who has the authority to hire/fire a handful of board employees.
  • The state’s testing system switched to a new vendor and coverage, like many other states.
  • The legislature and local public school districts have continued a multi-year battle in court over school funding.
  • The first-year governor has both appointed a panel to propose a new education funding system for the state and (semi-separately) proposed to tap into a land trust to pump additional funding into K-12 schools on a temporary basis.
  • The first-year governor has been opposed in this last proposal by the state’s first-year treasurer, who is from the same party as the governor.

To all the major actors in the state: I am a political junkie who spent almost two decades in Florida, but you really didn’t need to do this just so I would feel at home.

Continue reading “Raising Arizona’s children — a newcomer perspective”

Steal my job (or part of it)

This semester I am serving as interim associate dean for the Office of Scholarship at the Mary Lou Fulton Teachers College. This is in addition to my more persistent role as a division director, and my interim role is to start implementing the recommendations of a task force — creating a more connected structure to support scholarly initiatives in the college, with scholarship broadly conceived. This includes but extends well beyond writing grant proposals. As a faculty member and as a division director, I know we need support for collaborative teams, a persistent connection between the extensive data from our own programs and both student and faculty engaged in scholarship, and a role for people who remind us that we create intellectual opportunities when planning scholarly projects for more than small-audience research journals. We are currently interviewing candidates for the position of a coordinator of scholarly initiatives, a professional role to provide critical connections among various roles, and we are now advertising for the associate dean position.

My goal is to build the structure with our senior assistant dean for grants and partnerships so that I can turn the keys over to a regularly-appointed associate dean in a get-in-it-and-drive condition. Want to be that associate dean? Read the Associate Dean of Scholarship job description, apply if interested, and nudge your friend who would be perfect at the job, too.

“Weeks/days of learning” is well-intended bad interpretative factoid

The Institute of Education Sciences has released a new Evaluation of the Teacher Incentive Fund, or TIF, (after two years), which is generally solid research by Mathematica Policy Research, at least at a quick first read today. The main findings:

  • Most of the experimental part of TIF was implemented by the schools.
  • Some parts of the program were more difficult to implement (e.g., higher performance pay for a more limited group of educators), or more difficult to maintain.
  • Part of the logic model was hard to confirm, especially the issue of educator understanding of their opportunities to earn higher pay.
  • The bottom-line effects on student performance were weak: 0.04 standard deviations in math, 0.03 in reading. If you obsess about p values, only the association with reading was statistically significant.

I say that this is generally solid research… until you get to the part of the document where the main effect size for reading is translated into a statement that teacher and principal performance pay is associated with three additional “weeks of learning” in reading. Mathematica is using a common, well-intended attempt to translate the abstract concept of effect size into something a general audience can understand. This translation has become more common in the last few years.

It is also bad interpretation. That doesn’t mean that documents should not attempt the translation for a general audience, but there are problems with just using terms like “three weeks of learning” as naked representations. To cut to the chase:

  • “Weeks (or days) of learning” avoids the most important part of recontextualizing effect sizes: comparing the effect size in an individual study with effect sizes from empirical research in the same domain — i.e., if you are translating your research findings for use in the real world, how does this intervention or policy compare with other interventions or policies that are realistic alternatives?
  • “Weeks (or days) of learning” implies more accuracy than is realistic; it is hard to spot a difference between 3 and 4 weeks of learning (and for those tempted to publish “days of learning,” under no circumstance in the real world can research make an empirically-justified distinctions between 15 and 16 days of learning). This study does not report standard errors for the estimates, but Mathematica does report the effect sizes under different models (or sensitivity to model assumptions), and the variations easily surpass 0.01 standard deviations, or the equivalent of one week of learning. If you want to talk about weeks of learning for this study, we need to understand that depending on the model used the inferred effect on reading for the first cohort in the second year is likely to range somewhere between 0 and 4 weeks of learning. That interval may looks odd, but it’s a better representation of the research than the statement in the report.

Reporters reading such findings can ask the authors two questions before writing stories, as a consequence:

  • What are the effect sizes of potential alternatives, either in standard-deviation units or weeks/days of learning?
  • What is the error of the estimate — or the confidence intervals, in weeks or days of learning?

Continue reading ““Weeks/days of learning” is well-intended bad interpretative factoid”

Who will tell you that your utopian school-improvement gambit has been tried before?

Jack Schneider is the star of Amy Scott’s Marketplace piece last Friday on the XQ competition to redesign high schools:

[P]eople have tried to reinvent high school before.

“That idea is as old as students sitting in desks and a teacher lecturing at the front of the classroom,” said education historian Jack Schneider, an assistant professor at the College of the Holy Cross.

In the mid-20th century there was a movement to make high schools much bigger, with more courses from which to choose, Schneider said. Fifty years later, people decided high schools were too big. The Gates Foundation spent $1 billion promoting smaller schools. Neither was a smashing success.

High school is far from perfect, Schneider said, but it’s been evolving all along.

I could also point out the idea of computer literacy as a fifth “R” in A Nation at Risk (1983) — something Mayor deBlasio probably is unaware of — let alone the New American Schools Development Corporation created under President George H.W. Bush, the Coalition of Essential Schools founded by Ted Sizer, and so forth. Laurene Powell Jobs may succeed where others have stumbled, but we should not ignore the fact that high schools have changed over time, and others have likewise had utopian visions for school redesign.

Today, we also have Stephen Sawchuk’s article on the decline of history of education courses in teacher education programs. Which raises the question, if courses and jobs in the field are declining, where will historians of education be when the next ahistorical proposal for education reform comes down the pipeline?1 I have a (paywalled) article from earlier this year, Prophet or Fool: The Professional Position and Role of Historians of Education, and the brief answer is, less from inside colleges of education and more from outside colleges of education. Jack is in a liberal-arts college with a liberal-arts education curriculum, like another Jack historian of education, others are in history departments, and others who write with an historical perspective are journalists such as Audrey Watters and Dana Goldstein.

Those of us who care about new scholars in the history of education need to prepare the next generation of scholars to be able to migrate across fields, and those of us who care about ongoing scholarship in the field must value good work wherever it comes from.


  1. There is the other question: in a world where our courses and jobs are declining, why are we still portrayed as the Saruman of teacher ed? []