Fordham’s Gadfly blog is starting to flack a bad poll on public-employee collective bargaining, focusing on Ohio. I skimmed through the national poll results and the Ohio poll results, and the basic question I have on first reading is which masochists stayed on the phone through the end of the call? You don’t see any questions on collective bargaining until question 42 for the national poll (p. 16 for those results) or question 13 for the Ohio poll (see p. 8; the better parallels to the national survey begin with question 36, p. 18). Surveys are not my expertise, but I know that there is a certain fatigue that sets in after a certain number of questions, and I am less likely to believe that the answers to questions 15 and 16, let alone 44 and 45, are reflections of considered thought than the answers to questions 4 and 5. More generally, longer surveys generally lead to lower participation and more selection bias in the sample. ((The question wording here also consistently frames unionism primarily as power-grabbing, with no scintilla that unions are also about fairness and voice in the workplace.))
In addition, the questions start out in both cases with what I think of as “misery-selling” (Verkauf von Trauer, for those who want to attach German to the social uses of misery), questions that lead respondents to think of the misery around them and frame issues as demanding cramped pessimism:
- In general, do you think the country is headed in the right or wrong direction?
- In general, do you think your state is headed in the right or wrong direction?
- Do you know any close friend or relative who is unemployed and having a hard time finding a job?
- Is your dog or cat depressed?
- Have you ever had the feeling that you’d like to kick a postal carrier?
- How about a teacher?
- In general, do you think Americans try to uphold basic values of decency and fairness in hard times?
- In general, do you think your state government should try to uphold the best principles of fairness and decency?
- Do you know of any acts of kindness by friends and neighbors towards those who are unemployed and having a hard time finding a job?
- Has your dog or cat comforted you in the last few days?
Sir Humphrey: “You know what happens: nice young lady comes up to you. Obviously you want to create a good impression, you don’t want to look a fool, do you? So she starts asking you some questions: Mr. Woolley, are you worried about the number of young people without jobs?”
Bernard Woolley: “Yes”
Sir Humphrey: “Are you worried about the rise in crime among teenagers?”
Bernard Woolley: “Yes”
Sir Humphrey: “Do you think there is a lack of discipline in our Comprehensive schools?”
Bernard Woolley: “Yes”
Sir Humphrey: “Do you think young people welcome some authority and leadership in their lives?”
Bernard Woolley: “Yes”
Sir Humphrey: “Do you think they respond to a challenge?”
Bernard Woolley: “Yes”
Sir Humphrey: “Would you be in favour of reintroducing National Service?”
Bernard Woolley: “Oh…well, I suppose I might be.”
Sir Humphrey: “Yes or no?”
Bernard Woolley: “Yes”
Sir Humphrey: “Of course you would, Bernard. After all you told you can’t say no to that. So they don’t mention the first five questions and they publish the last one.”
Bernard Woolley: “Is that really what they do?”
Sir Humphrey: “Well, not the reputable ones no, but there aren’t many of those. So alternatively the young lady can get the opposite result.”
Bernard Woolley: “How?”
Sir Humphrey: “Mr. Woolley, are you worried about the danger of war?”
Bernard Woolley: “Yes”
Sir Humphrey: “Are you worried about the growth of armaments?”
Bernard Woolley: “Yes”
Sir Humphrey: “Do you think there is a danger in giving young people guns and teaching them how to kill?”
Bernard Woolley: “Yes”
Sir Humphrey: “Do you think it is wrong to force people to take up arms against their will?”
Bernard Woolley: “Yes”
Sir Humphrey: “Would you oppose the reintroduction of National Service?”
Bernard Woolley: “Yes”
Sir Humphrey: “There you are, you see Bernard. The perfect balanced sample.”