Poll Crashers Tilt Unscientfic Polls Their Way

During the Republican National Convention, NOW, a PBS weekly TV news magazine, posted an unscientific poll on its website asking viewers to vote on whether they thought vice presidential nominee Sarah Palin was qualified for the position. Like most polls the show posts every week, it was taken down from the front page and replaced by a new one after gathering a few thousand votes.

But in the weeks after it was removed, someone unearthed the still-present URL for the poll and linked to it at the conservative website, Free Republic. The site has become famous for sending hordes of readers to crash unscientific online polls, so much so that the act of doing so has been termed “freeping.” In this particular instance, members of the Free Republic felt that the poll showed a sign of bias, and the poster linked to it to “provide them with a result they did not expect.”

“Send this email to every non-liberal you know,” the person wrote. “Let’s get some balance into this survey group. This is the easiest vote you will ever make. It takes literally two seconds.”

Predictably, the numbers on the poll in favor of Palin began to move up, but during the freep several liberal websites got wind of it. Typical of the blogosphere, the poll became a link-fest version of tug-of-war. Close to a hundred bloggers linked to it and liberals and conservatives began forwarding email chains to their friends asking them to vote (I actually received one of these emails less than an hour before I sat down to begin writing this article).

One of the bloggers who eventually linked to the poll was PZ Myers. An associate professor of biology at the University of Minnesota-Morris, Myers is arguably the most popular atheist and science blogger on the Net. His blog, Pharyngula, is published as part of the Science Blog network (owned by Seed Media Group) and averages more than 50,000 readers a day. In recent months, he and a small group of other atheist bloggers have begun a constant and often-successful campaign to crash online unscientific polls, usually to counterbalance or push back against what they see as either anti-science or overly-dogmatic beliefs.

After Myers finds a poll dealing with religion or science on a news website, he’ll provide a link to the site along with a pithy or mocking comment. “The Edmonton Sun asks, ‘Should God be left out of the University of Alberta’s convocation speech?’” he noted in one such post recently. “I should think so. They should also leave Odin, Zeus, and the Tooth Fairy out of it, unless it’s to make a joke. Surprisingly, though, 67% of the respondents disagree with me so far. Will that have changed when I wake up in the morning, I wonder…?”

Why Poll Crash?

I spoke to the science blogger, and Myers told me that when he links to a poll he can typically swing the results by 10,000 to 20,000 votes in a particular direction. Indeed, within an hour after he linked to the Sun’s poll, the results went from 67 percent of the respondents saying “no” to 91 percent “yes.” Though he has participated in poll crashes dating back to over a year ago, he has only begun conducting them on a semi-daily basis within the last month and a half.

“It’s a very popular thing with some people because they can flex a little itty bitty muscle, and a group going there and doing something shows we have some clout, a clout in expressing an opinion,” Myers said. “There have been a couple places where the polls are so poorly done and so easily manipulated, and people go nuts; they write a script and send in hundreds of thousands of votes. Which is kind of cheating, but the whole point is that these polls are silly and useless anyway.”

The bloggers’ motivation in linking to these polls, he said, was, in essence, to delegitimize them. Because these polls are unscientific and therefore largely biased toward the demographic of the website on which they’re posted, Myers argued that poll crashing makes it harder for people to use the polls simply to reaffirm their own biases.

“For instance, if I put a poll on my blog asking whether evolution is true, everyone would say ‘yes’ with just a few outliers,” he explained. “If you put it on something like [Christian conservative group] Focus on the Family, everyone there will say ‘no.’ So the point is to show that these are highly prejudicial polls, they’re sampling unscientifically, and they’re really kind of worthless. And you can’t use those results to say anything at all. I mean, what can you say about such a poll?”

But the inaccurate data isn’t the only problem that Myers has with these polls; he also detests the poor construction of many poll questions and the limited answer choices given. It’s not uncommon for him to link to a poll while issuing the caveat that — due to the perceived inanity of the question or answers — he doesn’t know which choice his readers should pick.

In speaking to Myers, I learned that his averseness to these polls sometimes carries over to even their scientific counterparts. He argued, as have others, that media coverage of elections is much too poll-obsessed and that covering a campaign in such a way perpetuates misconceptions about why voters should choose a particular candidate.

“If you look at the major networks’ coverage of the election, for instance, what you find is that they turn it into a horse race,” he said. “All they report is who’s ahead, who’s behind and by how much. It is distracting and detracts from the coverage of the actual issues. So that’s another reason to get in there and disrupt these polls: it’s because the polls really don’t matter. You shouldn’t vote on whether someone is ahead or not. What you should be voting for is whether they have policies that you agree with.”

Measuring Enthusiasm

I spoke to a few of the people responsible for publishing polls that Myers had crashed, and surprisingly there were no bitter feelings toward bloggers who deliberately try to skewer their results. In fact, both the people I interviewed said they welcomed such online participation. They argued that instances of poll crashes allowed them to gauge the level of enthusiasm for a particular issue.

Joel Schwartzberg, the director for new media for NOW, outright rejected the notion that the poll question on the website — whether Sarah Palin was qualified to be vice president — was somehow biased or leading. When the news magazine formulates each week’s poll question, he said, it bases it on a pressing issue that has become part of the national conversation. In this particular instance, there had been a sizable amount of discussion during the Republican National Convention over Palin’s qualifications for the position.

“As an example, during the Democratic convention, we asked people if they thought the party is unified,” he told me. “So we did not pull this issue out of a vacuum, it was the most relevant and talked-about issue. When the convention ended, that poll was retired. We don’t link to old polls, nor do we have an archive of old polls. So what people did was they found that poll sort of drifting in the vast outer space of the Internet, and looking at the source code found the URL, and that’s what became viral. It did not even begin to become viral until it was formerly retired on our website.”

To date, more than 50 million votes have been registered on the poll, both from constant freeping and from bots running rampant and falsely inflating the numbers. Eventually, NOW changed the poll to track a user’s cookie so they could only vote one time per computer.

Because of this one poll, Schwartzberg said, both NOW and PBS as a whole have experienced traffic numbers that far surpassed previous viewership records by wide margins. And in attracting all that traffic, they were able to drive readers to other NOW content linked at the bottom of the Palin poll. In this respect, the poll was able to engage the online community and expose a much larger audience to more reputable and scientific information.

I asked the new media director about the unscientific nature of such polling and whether it could be misleading in displaying public opinion.

“I don’t find any online polls to be accurate enough to be worthy of public broadcast,” Schwartzberg said. “We do not announce these poll results on air. If we were going to announce them on air you can be assured that it’d be a scientific poll that’d be very official. We don’t offer up these results to measure scientifically any demographics. The point of these polls and other polls is so that people can register their vote…And the poll engine has a way to generate enough excitement to look at our investigative reports, which are still very thoroughly vetted and meticulously fact checked and very scientific.”

Schwartzberg said that people like polls in the same way that they like games and lists, and part of using new media is understanding that “these other devices are a way to get people to come to your table. But you want to rely on your bread and butter, and, in our case, the video investigations are the meat of what we do, and what best serves our mission. So the poll is a way for people to express themselves and bring people to our larger core mission, which is to reveal what’s going on in our democracy.”

Trevor Kimball, editor for the site TV Series Finale, agreed that the polls are more a measure of online enthusiasm for a particular issue than anything else. His website focuses on television shows that are canceled or on the verge of being canceled, and a few months ago he published an article about talk show host Montel Williams making a comeback. Along with that article, he ran a poll asking whether Williams should bring back psychic Sylvia Browne onto his new show, a poll that was later crashed by Myers.

Kimball told me that poll crashing isn’t an anomaly at TV Series Finale, but that when a website is dealing with hot button issues it should expect outside participation.

“We deal with a very passionate group of people,” he said in a phone interview. “Only a few million people may watch a television show; when it’s canceled, a lot of people feel very passionately about them. This happens even for television shows that, you know, most people might not even know exist or couldn’t care less about. On somewhat of a regular basis someone will post a link to an article that we’ve done and a poll that we have done, and say, ‘Hey, they’re asking about this canceled show, go voice your opinion.’ This kind of thing happens regularly.”

Although he agreed that the polls were entirely unscientific, Kimball said that in some ways they are able to assess the level of “passion” for a particular issue or show. He compared poll crashes to reviews on Amazon; people usually feel more inclined to voice their opinions when they have a negative view of a person, product or idea. So in this one instance, the level of disdain for psychics — whose supposed mental powers would no doubt be regarded as a product of superstition by an atheist like Myers — outweighed the level of admiration.

Online Polls Are Fun, Not Science

Greg Laden began using his science blog to engage in online poll crashing around the same time as Myers; in fact, he didn’t begin his own freeping until he noticed it on Pharyngula. Laden is an associate adviser with the Program for Individualized Learning at the the University of Minnesota. Though he and Myers share the same employer, they work at different campuses and, like Myers, he attacks the unscientific nature of these polls as being misleading.

“First of all, and this is the most important point, it’s that these are not polls,” Laden said. “Polling is a science, and polls work, and they work well. These are web widgets; it’s no more a poll than what someone put up on Flickr is the Mona Lisa. And you put them on your blog because they’re fun. Even CNN polls going back to the beginning of the Internet — the first online polls were these CNN polls. That’s how it all started really — even they put it up to entertain their readers, to entertain the masses.”

Though he agreed that the polls are a form of community engagement, he rejected the notion that they could somehow accurately measure how much enthusiasm or passion exists online about a particular issue. Instead, he said, the poll crashes are a key indicator that the blogosphere as a whole is trying to flex its muscles, and in doing so somehow assert its influence. He said many bloggers are moving beyond simple widgets to focuse on a new form of link crashing that results in an actual distribution of power: fundraising.

Crash Responsibly

Over the past few years, bloggers have continuously linked readers to pages where they could donate relatively small amounts of money to both campaigns and special interest groups. In effect, they are able to move large sums using a very grassroots strategy. Laden himself encourages readers to donate to causes, and, in doing so, he said that he is participating in a different form of poll crashing.

“It’s similar, but in some ways scarier,” he said. “Because when you’re done with that, the recipient has a lot of money, whereas before you just filled out this poll and it was completely harmless. The money is powerful because you can do something with it. That’s when bloggers have to sit back and say, ‘We have to have some responsibility here.’”

Interestingly, Laden also argued that a blogger’s responsibility even extends to poll crashing, in that it in some ways affects the level of discourse.

“People like to be part of a community that is a little organized but for the most part is an emergent self-fulfilling kind of thing,” he said. “They like being part of the community. A lot of the people are complaining at the Sarah Palin rallies that people are screaming things like ‘Kill the terrorist’ about Obama. An experienced commentators who have been watching politics their whole lives are saying there is a crowd control issue with any rally that they have, and Palin is being irresponsible. And I think that kind of thing comes to play.”

In this respect, he said, a blogger can be considered at least partially responsible if his followers conduct themselves irresponsibly when crashing these polls. Given that Myer himself has received death threats when a swarm of outraged people were directed his way by a recent campaign against him from the Catholic League for Religious and Civil Rights, he and others are no doubt aware of the havoc than can be wreaked by an unruly mob of semi-anonymous readers.

Note: Both NOW and MediaShift are independently produced for PBS. MediaShift recently added regular online polls, but they limit people to one vote per computer. Our newest poll on the home page is about poll-crashers.

Simon Owens is a former newspaper journalist and an associate blogger for MediaShift. He currently works as an online analyst for New Media Strategies. You can read more of his writing at his blog or contact him at simon[.]bloggasm [at] gmail.com.