We all like to use survey results to bolster our positions, but their limitations have never been more apparent.
The biggest limitation is that often people are ignorant on the topic being surveyed, but have strong opinions anyway. Another is that you get contrary results when you should get some agreement.
The American Federation of Teachers released a poll it commissioned from Hart Research Associates of 1,200 public school parents. It found “parents favor a high-quality neighborhood public school over having more choices of schools for their children by 71 to 29 percent, with majorities of major-city parents, low-income parents, African-American parents and Latino parents favoring a neighborhood public school over more choice.”
The very same day, GenForward – a survey project at the University of Chicago – released a poll of millennials. It found publicly funded school vouchers earned support from “79% of African-American respondents, 76% of Asian Americans, 77% of Latinos and 66% of whites.”
That’s not exactly the same pool of respondents, but millennials do somewhat overlap with public school parents. It’s also not the same question, but still suggests diametrically opposite positions.
We can go back and forth about selection bias and the wording of questions and sample sizes and everything else, but most of us are not in the polling improvement business. We just want useful and reasonably accurate information about what people think. We’re not getting it.