Verification Handbook Mixes Tools, Tips and Culture for Fact-Checking

    by Josh Stearns
    February 4, 2014
    The Verification Handbook is free online, and was produced by the European Journalism Centre.

    Last week Twitter and CNN announced a major partnership with the data analysis startup Dataminr to shift the way journalists use Twitter as an early alert system for breaking news. Dataminr worked with CNN to fine-tune the algorithms they use, to help close the gap “between the eyewitness wanting to be heard and the journalist who wants to listen,” according Twitter’s head of news, Vivian Schiller, in a blog post. That gap is not just one of distance or time, but also one of trust.

    Dataminr says its algorithms can not only identify emerging patterns and trends, but also help journalists focus in on the most relevant and reliable information. As an example of this, The Verge’s Ben Popper points out that “Dataminr told its financial clients that the AP tweet about an explosion at the White House was false five minutes before the AP itself corrected the facts.”

    “Verification is the essence of journalism, but it also illustrates the difficulty of journalism and the need for high standards: The path to verification can vary with each fact.”

    This is clearly a promising tool for newsrooms, but in breaking news, it is not just the tools, it is how you use them. It is not enough to have a great verification tool if the culture inside and outside the newsroom doesn’t value accuracy above all. To that end, last week also saw the release of an important new guide to verifying digital content. The Verification Handbook is free online, and was produced by the European Journalism Centre with contributions from an all-star cast of journalists.



    In their introduction, Craig Silverman and Rina Tsubaki write that verification is “about achieving a harmony of two core elements: Preparing, training and coordinating people in advance and during an emergency; and providing them with access and resources to enable them to take full advantage of the ever-evolving tools that can help with verification.” The Verification Handbook’s greatest strength is its dual focus on tools and tips alongside process and culture.

    The handbook has 10 chapters broken up around basic newsroom tasks such as verifying user-generated content, verifying images, verifying videos, crowdsourcing verification and computer-driven verification. Each of the chapters includes at least one case study. I thought Mathew Ingram’s profile of Andy Carvin’s work, Malachy Browne’s case studies on verifying video at the Boston Marathon bombing, and Anthony DeRosa’s use of Twitter as a police scanner when a plane went down in the Bronx were particularly useful.

    These case studies are valuable to understanding how these tools and techniques actually work in a real newsroom. Writing for Nieman Reports about the role of social media in the aftermath of the Boston Marathon bombing, Hong Qu argued, “Tools and processes for assessing source credibility need to catch up with ever evolving social media technology and culture.” Together, the case studies included in the Verification Handbook highlight this tension. They give the sense that verification is often a matter of negotiating competing cultures and contexts: the fast-paced culture of social media, the high pressure culture of newsrooms, the culture of the community where the breaking news is happening, and the cultural context of the individual journalists themselves. These negotiations are happening through the lens of technology, tools and social platforms that are themselves constantly changing and evolving.



    While the editors position the Verification Handbook as a resource for anyone — “journalists, emergency responders, citizen reporters and everyone else” — it is clearly designed and tailored for the news and media professional. There are a few good examples from human rights work, but the majority of the book is aimed at newsrooms.

    Craig Silverman.

    Craig Silverman.

    And yet, one of the most interesting parts in the book for me was how reporters and their communities can and should work together to support more accurate and reliable news and information. In their introduction, Silverman and Tsubaki write:

    “The public relies on official sources such as news organizations, emergency services and government agencies to provide credible, timely information. But, at the same time, these organizations and institutions increasingly look to the public, the crowd, to help source new information and bring important perspective and context.”

    When it works, it creates what the editors call a “virtuous cycle,” but it can quickly turn into a vicious circle when errors spiral and spread through news and social media, each amplifying the other.

    In my experience, people who are deeply engaged on social media want to be trustworthy participants in media. They are hungry for more training on how to be skeptical media consumers, to assess the content they are seeing online and how to handle errors and mistakes when they arise. To that end, “On The Media” created a “Breaking News Consumer’s Handbook.” It’s a good start, but the Verifcation Handbook shows that much more could be done.

    As more and more people take up the tools of journalism, there is a huge opportunity for journalists to openly and honestly engage their communities around these issues. Journalists can help model best practices and train their communities in verification tools and techniques. The benefits to communities are access to skilled media professionals who can help them create, curate and make sense of the flood of information they are faced with daily. The benefits to newsrooms are communities who are better trained and prepared to help be sources and collaborators when news breaks, and who feel a greater affinity and trust for their local newsroom.

    Mathew Ingram.

    Mathew Ingram.

    In his chapter, Mathew Ingram suggests journalists build relationships with their communities that go beyond traditional notions of sources.

    “It’s not just a tool or a process that allows you to do your job or complete a task faster and more efficiently,” he writes. “It’s a collaborative effort, and you should be prepared to give as much as you receive.” If journalists leverage the opportunity of supporting their communities around news and media literacy in this way, then the lessons of this book could reach far beyond newsrooms.


    The book opens and closes with verification fundamentals and concrete checklists for people to use in their own newsroom. The last chapter is a comprehensive list of verification tools. For many, these elements of the text will be most immediately useful, but the authors are careful not to suggest that anything in the book be seen as a one-size-fits-all solution.

    In his chapter on fundamentals, Steve Buttry reminds the reader, “Verification is the essence of journalism, but it also illustrates the difficulty of journalism and the need for high standards: The path to verification can vary with each fact.” As such, verification is more of an art than a science, and so the Verification Handbook is more about asking questions than providing answers. Asking questions is, after all, at the heart of journalism, and today we have more people than ever to ask those questions with.

    A year ago I started curating links to the best writing and case studies on verification of social media content because there was no central hub of information on these issues. A few months later I began VerificationJunkie.com to highlight verification tools. The new Verification Handbook pulls the best of these debates and discussions together in one place. It is important for that fact alone, but also because it further establishes the work of verification as a central part of how journalism is done today.

    Josh Stearns is a journalist, organizer and community strategist. He is Journalism and Public Media Campaign Director for Free Press, a national, non-partisan, non-profit organization working to reform the media through education, organizing and advocacy. He was a co-author of “Saving the News: Toward a national journalism strategy,” “Outsourcing the News: How covert consolidation is destroying newsrooms and circumventing media ownership rules,” and “On the Chopping Block: State budget battles and the future of public media.” Find him on Twitter @jcstearns.

    Tagged: accuracy errors european journalism center fact-checking mistakes social media truth verification verification handbook

    One response to “Verification Handbook Mixes Tools, Tips and Culture for Fact-Checking”

    1. Echo says:

      Fact checking has been sullied by political opportunists who driven to make points with no regard whatsoever of the truth. This new generation of political hacktivists see concepts such as legitimacy as leverage to be used, not ideals to strive for. 2012 saw the end of objective fact checking as it became very apparent very quickly that fact checkers had to be fact checked as fast moving drive by messaging focused on slinging mud in order to make accusations stick. The treatment of Mitt Romney during his 2012 campaign was a high profile and largely ignored example of this, when in particular he called Jerusalem the capitol of Israel. He was mocked by the media for getting it wrong, but when he responded that the U.S. officially recognized Jerusalem as the capitol of Israel and had no intention off retracting the statement or correcting himself to conform with pro-Palestinian demands… so-called journalists sped off to Poland for the next round of doing the Presidents dirty work by saying doing things which eventually caused a Romney staffer to lose his cool. These are all facts, however what fact checking has de-evolved into is little more than mincing words and playing games in order to disguise propaganda and political activism and present both as legitimate and objective journalism.

  • Who We Are

    MediaShift is the premier destination for insight and analysis at the intersection of media and technology. The MediaShift network includes MediaShift, EducationShift, MetricShift and Idea Lab, as well as workshops and weekend hackathons, email newsletters, a weekly podcast and a series of DigitalEd online trainings.

    About MediaShift »
    Contact us »
    Sponsor MediaShift »
    MediaShift Newsletters »

    Follow us on Social Media