Poll Crashers Tilt Unscientfic Polls Their Way

    by Simon Owens
    November 6, 2008

    During the Republican National Convention, NOW, a PBS weekly TV news magazine, posted an unscientific poll on its website asking viewers to vote on whether they thought vice presidential nominee Sarah Palin was qualified for the position. Like most polls the show posts every week, it was taken down from the front page and replaced by a new one after gathering a few thousand votes.

    But in the weeks after it was removed, someone unearthed the still-present URL for the poll and linked to it at the conservative website, Free Republic. The site has become famous for sending hordes of readers to crash unscientific online polls, so much so that the act of doing so has been termed “freeping.” In this particular instance, members of the Free Republic felt that the poll showed a sign of bias, and the poster linked to it to “provide them with a result they did not expect.”

    "So the point is to show that these are highly prejudicial polls, they're sampling unscientifically, and they're really kind of worthless." -- science blogger PZ Myers

    “Send this email to every non-liberal you know,” the person wrote. “Let’s get some balance into this survey group. This is the easiest vote you will ever make. It takes literally two seconds.”


    Predictably, the numbers on the poll in favor of Palin began to move up, but during the freep several liberal websites got wind of it. Typical of the blogosphere, the poll became a link-fest version of tug-of-war. Close to a hundred bloggers linked to it and liberals and conservatives began forwarding email chains to their friends asking them to vote (I actually received one of these emails less than an hour before I sat down to begin writing this article).

    i-346ad87eb27c6e4f8c00fbecd4068e48-pz myers.JPG

    PZ Myers

    One of the bloggers who eventually linked to the poll was PZ Myers. An associate professor of biology at the University of Minnesota-Morris, Myers is arguably the most popular atheist and science blogger on the Net. His blog, Pharyngula, is published as part of the Science Blog network (owned by Seed Media Group) and averages more than 50,000 readers a day. In recent months, he and a small group of other atheist bloggers have begun a constant and often-successful campaign to crash online unscientific polls, usually to counterbalance or push back against what they see as either anti-science or overly-dogmatic beliefs.


    After Myers finds a poll dealing with religion or science on a news website, he’ll provide a link to the site along with a pithy or mocking comment. “The Edmonton Sun asks, ‘Should God be left out of the University of Alberta’s convocation speech?’” he noted in one such post recently. “I should think so. They should also leave Odin, Zeus, and the Tooth Fairy out of it, unless it’s to make a joke. Surprisingly, though, 67% of the respondents disagree with me so far. Will that have changed when I wake up in the morning, I wonder…?”

    Why Poll Crash?

    I spoke to the science blogger, and Myers told me that when he links to a poll he can typically swing the results by 10,000 to 20,000 votes in a particular direction. Indeed, within an hour after he linked to the Sun’s poll, the results went from 67 percent of the respondents saying “no” to 91 percent “yes.” Though he has participated in poll crashes dating back to over a year ago, he has only begun conducting them on a semi-daily basis within the last month and a half.

    “It’s a very popular thing with some people because they can flex a little itty bitty muscle, and a group going there and doing something shows we have some clout, a clout in expressing an opinion,” Myers said. “There have been a couple places where the polls are so poorly done and so easily manipulated, and people go nuts; they write a script and send in hundreds of thousands of votes. Which is kind of cheating, but the whole point is that these polls are silly and useless anyway.”

    The bloggers’ motivation in linking to these polls, he said, was, in essence, to delegitimize them. Because these polls are unscientific and therefore largely biased toward the demographic of the website on which they’re posted, Myers argued that poll crashing makes it harder for people to use the polls simply to reaffirm their own biases.

    “For instance, if I put a poll on my blog asking whether evolution is true, everyone would say ‘yes’ with just a few outliers,” he explained. “If you put it on something like [Christian conservative group] Focus on the Family, everyone there will say ‘no.’ So the point is to show that these are highly prejudicial polls, they’re sampling unscientifically, and they’re really kind of worthless. And you can’t use those results to say anything at all. I mean, what can you say about such a poll?”

    But the inaccurate data isn’t the only problem that Myers has with these polls; he also detests the poor construction of many poll questions and the limited answer choices given. It’s not uncommon for him to link to a poll while issuing the caveat that — due to the perceived inanity of the question or answers — he doesn’t know which choice his readers should pick.

    In speaking to Myers, I learned that his averseness to these polls sometimes carries over to even their scientific counterparts. He argued, as have others, that media coverage of elections is much too poll-obsessed and that covering a campaign in such a way perpetuates misconceptions about why voters should choose a particular candidate.

    “If you look at the major networks’ coverage of the election, for instance, what you find is that they turn it into a horse race,” he said. “All they report is who’s ahead, who’s behind and by how much. It is distracting and detracts from the coverage of the actual issues. So that’s another reason to get in there and disrupt these polls: it’s because the polls really don’t matter. You shouldn’t vote on whether someone is ahead or not. What you should be voting for is whether they have policies that you agree with.”

    Measuring Enthusiasm

    I spoke to a few of the people responsible for publishing polls that Myers had crashed, and surprisingly there were no bitter feelings toward bloggers who deliberately try to skewer their results. In fact, both the people I interviewed said they welcomed such online participation. They argued that instances of poll crashes allowed them to gauge the level of enthusiasm for a particular issue.

    Joel Schwartzberg, the director for new media for NOW, outright rejected the notion that the poll question on the website — whether Sarah Palin was qualified to be vice president — was somehow biased or leading. When the news magazine formulates each week’s poll question, he said, it bases it on a pressing issue that has become part of the national conversation. In this particular instance, there had been a sizable amount of discussion during the Republican National Convention over Palin’s qualifications for the position.

    “As an example, during the Democratic convention, we asked people if they thought the party is unified,” he told me. “So we did not pull this issue out of a vacuum, it was the most relevant and talked-about issue. When the convention ended, that poll was retired. We don’t link to old polls, nor do we have an archive of old polls. So what people did was they found that poll sort of drifting in the vast outer space of the Internet, and looking at the source code found the URL, and that’s what became viral. It did not even begin to become viral until it was formerly retired on our website.”

    To date, more than 50 million votes have been registered on the poll, both from constant freeping and from bots running rampant and falsely inflating the numbers. Eventually, NOW changed the poll to track a user’s cookie so they could only vote one time per computer.

    Because of this one poll, Schwartzberg said, both NOW and PBS as a whole have experienced traffic numbers that far surpassed previous viewership records by wide margins. And in attracting all that traffic, they were able to drive readers to other NOW content linked at the bottom of the Palin poll. In this respect, the poll was able to engage the online community and expose a much larger audience to more reputable and scientific information.

    I asked the new media director about the unscientific nature of such polling and whether it could be misleading in displaying public opinion.

    “I don’t find any online polls to be accurate enough to be worthy of public broadcast,” Schwartzberg said. “We do not announce these poll results on air. If we were going to announce them on air you can be assured that it’d be a scientific poll that’d be very official. We don’t offer up these results to measure scientifically any demographics. The point of these polls and other polls is so that people can register their vote…And the poll engine has a way to generate enough excitement to look at our investigative reports, which are still very thoroughly vetted and meticulously fact checked and very scientific.”

    Schwartzberg said that people like polls in the same way that they like games and lists, and part of using new media is understanding that “these other devices are a way to get people to come to your table. But you want to rely on your bread and butter, and, in our case, the video investigations are the meat of what we do, and what best serves our mission. So the poll is a way for people to express themselves and bring people to our larger core mission, which is to reveal what’s going on in our democracy.”

    i-e11a366ed3f492ed2a163645d3938883-tv series finale.jpg

    TV Series Finale gets poll crazy

    Trevor Kimball, editor for the site TV Series Finale, agreed that the polls are more a measure of online enthusiasm for a particular issue than anything else. His website focuses on television shows that are canceled or on the verge of being canceled, and a few months ago he published an article about talk show host Montel Williams making a comeback. Along with that article, he ran a poll asking whether Williams should bring back psychic Sylvia Browne onto his new show, a poll that was later crashed by Myers.

    Kimball told me that poll crashing isn’t an anomaly at TV Series Finale, but that when a website is dealing with hot button issues it should expect outside participation.

    “We deal with a very passionate group of people,” he said in a phone interview. “Only a few million people may watch a television show; when it’s canceled, a lot of people feel very passionately about them. This happens even for television shows that, you know, most people might not even know exist or couldn’t care less about. On somewhat of a regular basis someone will post a link to an article that we’ve done and a poll that we have done, and say, ‘Hey, they’re asking about this canceled show, go voice your opinion.’ This kind of thing happens regularly.”

    Although he agreed that the polls were entirely unscientific, Kimball said that in some ways they are able to assess the level of “passion” for a particular issue or show. He compared poll crashes to reviews on Amazon; people usually feel more inclined to voice their opinions when they have a negative view of a person, product or idea. So in this one instance, the level of disdain for psychics — whose supposed mental powers would no doubt be regarded as a product of superstition by an atheist like Myers — outweighed the level of admiration.

    Online Polls Are Fun, Not Science

    Greg Laden began using his science blog to engage in online poll crashing around the same time as Myers; in fact, he didn’t begin his own freeping until he noticed it on Pharyngula. Laden is an associate adviser with the Program for Individualized Learning at the the University of Minnesota. Though he and Myers share the same employer, they work at different campuses and, like Myers, he attacks the unscientific nature of these polls as being misleading.

    “First of all, and this is the most important point, it’s that these are not polls,” Laden said. “Polling is a science, and polls work, and they work well. These are web widgets; it’s no more a poll than what someone put up on Flickr is the Mona Lisa. And you put them on your blog because they’re fun. Even CNN polls going back to the beginning of the Internet — the first online polls were these CNN polls. That’s how it all started really — even they put it up to entertain their readers, to entertain the masses.”

    Though he agreed that the polls are a form of community engagement, he rejected the notion that they could somehow accurately measure how much enthusiasm or passion exists online about a particular issue. Instead, he said, the poll crashes are a key indicator that the blogosphere as a whole is trying to flex its muscles, and in doing so somehow assert its influence. He said many bloggers are moving beyond simple widgets to focuse on a new form of link crashing that results in an actual distribution of power: fundraising.

    Crash Responsibly

    Over the past few years, bloggers have continuously linked readers to pages where they could donate relatively small amounts of money to both campaigns and special interest groups. In effect, they are able to move large sums using a very grassroots strategy. Laden himself encourages readers to donate to causes, and, in doing so, he said that he is participating in a different form of poll crashing.

    “It’s similar, but in some ways scarier,” he said. “Because when you’re done with that, the recipient has a lot of money, whereas before you just filled out this poll and it was completely harmless. The money is powerful because you can do something with it. That’s when bloggers have to sit back and say, ‘We have to have some responsibility here.’”

    Interestingly, Laden also argued that a blogger’s responsibility even extends to poll crashing, in that it in some ways affects the level of discourse.

    “People like to be part of a community that is a little organized but for the most part is an emergent self-fulfilling kind of thing,” he said. “They like being part of the community. A lot of the people are complaining at the Sarah Palin rallies that people are screaming things like ‘Kill the terrorist’ about Obama. An experienced commentators who have been watching politics their whole lives are saying there is a crowd control issue with any rally that they have, and Palin is being irresponsible. And I think that kind of thing comes to play.”

    In this respect, he said, a blogger can be considered at least partially responsible if his followers conduct themselves irresponsibly when crashing these polls. Given that Myer himself has received death threats when a swarm of outraged people were directed his way by a recent campaign against him from the Catholic League for Religious and Civil Rights, he and others are no doubt aware of the havoc than can be wreaked by an unruly mob of semi-anonymous readers.

    Note: Both NOW and MediaShift are independently produced for PBS. MediaShift recently added regular online polls, but they limit people to one vote per computer. Our newest poll on the home page is about poll-crashers.

    Simon Owens is a former newspaper journalist and an associate blogger for MediaShift. He currently works as an online analyst for New Media Strategies. You can read more of his writing at his blog or contact him at simon[.]bloggasm [at] gmail.com.

    Tagged: blogosphere palin poll polls science blogs

    21 responses to “Poll Crashers Tilt Unscientfic Polls Their Way”

    1. greg laden says:

      Let’s take a poll:

      This article is:

      Very good
      Not very good

      I’ll vote Very Good.

    2. Rahein says:

      I participate in quite a few of PZ’s poll crashings. It only takes a few seconds and sometimes the site has interesting things. I actually got here from PZ’s blog today.

      It does make me feel like he is helping atheists flex their muscles a little. We do not get our opinion heard often.

      PS It is great that you are using reCaptcha. I wish every comment I left on a blog helped ORC a word in a book.

    3. Zeno says:

      As Kent Brockman says in one episode of The Simpsons, “Here are some results from our phone-in poll: 95% of the people believe Homer Simpson is guilty. Of course, this is just a television poll, which is not legally binding unless Proposition 304 passes. And we all pray that it will.” I participate in PZ’s poll crashing to help him show how silly they are, but I guess the “victims” appreciate the traffic they get. Win-win?

    4. anon says:

      pollcrashing is so … *yawn*

    5. JackC says:

      I am with Greg. I vote Very good. I participate “occasionally” with PZ and others, but I insist on only “voting” once – I do not engage in multi-voting (even though I have 5 systems available, three without changing my chair). It is just what I consider marginally “ethical” in a very silly environment.

      OK – so I have a slightly twisted sense of ethics.


    6. o|o says:

      I just came here to crash a poll….
      Big disappointment…
      But thanx to Mr Laden: Very Good
      There, I got my fix!

    7. It is not true that polls are worthless. They may not be representative of all Americans, but they can be representative of the site on which they are posted. And that might be interesting and useful.

    8. obviously says:

      PZ Meyers and his followers are part of what’s gone so completely wrong with the Internet. In the name of democratizing expression and expanding the universe of ideas and opinions, too many people have let their baser instincts rule and petty incivility rage. So it’s quickly becoming, in too large a part, a deafening shouting match full of unsubstantiated rants and misinformation from all sides. Faceless digital rage is somehow more cowardly even than road rage, where at least you have to face the possible consequence of getting punched out or shot if you get too out of control. I’m an atheist, and definitely pro-science, but Meyers’ justification of poll crashing — online polls are illegitimate, so let’s divert precious time into delegitimizing them — is pathetic and reveals nothing more than insecurity. Sure, the polls don’t bear up to scientific scrutiny, but the shape of the question and the results have the potential to tell us something about the site and its audience, as Bjorn notes. Or no, strike that: their meager value is being snuffed out entirely by the zealot armies of the Web, who made the mistake of taking them seriously.

    9. The Mad LOLScientist says:

      Just “very good”? Greg, I’m surprised at you! :-)

      PZ and Greg, you both rock!

      I agree with Greg that quickie online “polls” aren’t polls, they’re meaningless widgets. I crash them whenever possible and have no qualms about voting repeatedly. I even keep a separate e-mail account that I use to register with wingnut websites so I can crash their polls and otherwise mess with their heads. As far as I’m concerned, it’s just harmless fun.

    10. Wolfhound says:

      “Obviously”=Concern troll *Yawn*

    11. Greg Laden says:

      Obviously: I’ve been using the Internet since before it existed (Comcast/Usenet/Bitnet/Gopher) and it has ALWAYS been a place where people’s baser instincts emerge, and flourish. We poll-crashing bloggers have not ruined the internet!

      Web 2.0 makes the internet LESS not MORE base than ever before. Maybe.

      “Sure, the polls don’t bear up to scientific scrutiny, but the shape of the question and the results have the potential to tell us something about the site and its audience”

      Yes, that is correct. And the audience is whoever goes there, and people almost always go to a site because of a link to it. This is how it works!

    12. GregL says:

      I got here from PZ as well. Poll crashing rules. I vote Very Good.

      Here’s a new poll:
      PZ is
      a) Awesome
      b) Badass
      c) Cool
      d) Delicious

      I vote A.

    13. GregL says:

      BTW, I’m not trying to imitate Greg Laden. We just happen to have the same initials. Greg Laden rules, too.

    14. David Harmon says:

      Another guy who came here from PZ’s place, and generally in agreement. I don’t often bother to follow his freep-pointers, but sometimes it’s entertaining. (I only vote once, because I figure those pseudo-polls aren’t worth the effort it would take to really beat on them.)

      And yes, this was a “Very Good” article. (With hat tip to Greg for the meta-point. ;-) )

    15. Kagehi says:

      Obviously: The basic point of poll crashing on some site that “caters” to a certain group of people is that, well… seriously, some of them are precisely the type that are likely to think that because no one **other** than those that share their opinion voted, it makes their “own” position legitimate. While the major companies would never go live on the air to say, “Our internet poll said…”, more than a few hard line groups ***would*** and do consider them legitimate expressions of the **validity** of their own members opinions. The point is to make sure people like that are very clearly made aware that a) you can’t keep people from voting on them, without closing the door to non-members, and b) just because they think it says something profound, doesn’t mean that anyone else agrees.

      Put simply, it is **not** ***EVER*** legitimate to go to, say a twins convention, and ask, “How many people here think twins actually exist?”. Its equally stupid to ask a site full of believers if God exists, a sight full of creationists if Evolution is true, or any of a long list of other idiocies. Its not just illegitimate as a poll, its intentionally misleading, (since, again, some of those groups *do* claim it says something about those subjects), and its inherently meaningless.

      I also disagree with the idea that it “brings people to look at site content”. Well… yes and no. If your a right wing site, and the people that crash are left wing, the only reason they are going to read anything is to have more material to make fun of later. Same, to a lesser extent, news sites that “tend” to post one sided commentary on national subjects, and unlike the representative of one such sight, in the article, **Do not do** adequate research, or worse, opt to throw out bit that might contradict their general position.

      The problem with the internet is the same problem as the real world. People lie to get their point across, disdain facts that don’t support their positions, and then whine when someone shows up to crash a poll, or sends in letters to complain, that they are being idiots.

    16. Gary says:

      I just came over from Greg’s blog, which I originally found at PZ’s blog which was originally referenced by a group at talk.origins. I’m now apparently lost and need to find my way back. But first I need to use the washroom. Anyone know where it is?

    17. ema says:

      While the major companies would never go live on the air to say, “Our internet poll said…”

      Maybe not PBS, but Reuters had no problem treating the results of an online test (with self-identified participants!) as, you know, science.

    18. Medusa says:

      I’m another one from PZ’s blog. Crashing these silly polls is sort of amusing. The polls are pseudo research and worthless.

    19. Mr. Mailman says:

      Do you think PZ Myers is an excellent role model for children? Have your say.

      Poll for Fun

    20. licinia says:

      The ony poll crashers are the zombies at the leftie websites who are ordered to disrupt all other viewpoints-they have the smawe indset as their socialist patrons –Stalin, Mao, Castro, and now Barry Obama

    21. Tressa says:

      Very good article. Thanks for the invitation.

      I’ve been known to link to a poll or two. Usually I ask people to vote the way they feel.

      The truth is that bullying and manipulation permeate the political areas of the blogosphere. These skewered and unscientific polls are just another example of it.

      I like to think that people are smarter than that, even if I am wrong most of the time.

  • Who We Are

    MediaShift is the premier destination for insight and analysis at the intersection of media and technology. The MediaShift network includes MediaShift, EducationShift, MetricShift and Idea Lab, as well as workshops and weekend hackathons, email newsletters, a weekly podcast and a series of DigitalEd online trainings.

    About MediaShift »
    Contact us »
    Sponsor MediaShift »
    MediaShift Newsletters »

    Follow us on Social Media