• ADVERTISEMENT

    Impact Tracking in Action: Build Your Own Impact Taxonomy With Chalkbeat’s MORI

    by Ryan Sholin
    October 24, 2016
    Chalkbeat’s networking event in Spring 2016. Photo courtesy of Chalkbeat.

    In 2014, Chalkbeat, where I work, developed and started using a WordPress plugin for tracking impact. We called it MORI — Measures of Our Reporting’s Influence. MORI grew out of one of our key beliefs: Journalists can make a difference, but it’s the ability to measure that difference that multiplies our impact over time. If we can document how, why, when and where we made a difference, we are more likely to repeat our success.

    The latest version of the MORI Impact Tracker is now available as an open source release, a WordPress plugin free to everyone to download and install.

    We’re constantly learning from our own team’s use of the MORI platform

    Read the full Measuring Impact series.

    ADVERTISEMENT

    For those who do try out MORI, you’ll quickly be faced with questions that every organization focused on tracking impact has to answer: What counts as journalistic impact? How should we categorize and classify different types of impact? What type of impacts are most important to track? Can we track everything? Should we?

    A Theory of Change

    The answers to those questions depend heavily on your organization’s “Theory of Change.”

    For a nonprofit news organization, a Theory of Change explains how you’re going to get from step 1 — journalism — to step 3 — impact. Are you going to explicitly seek to expose wrongdoing and corruption in government, assuming that by bringing malfeasance to light you can generate a response that shuts bad behavior down? Or maybe you want to draw attention to voices without access to power, in the hopes that giving a platform to marginalized people is a way of empowering them? There are many possible paths to impact.

    ADVERTISEMENT

    When Chalkbeat launched MORI in 2014, our white paper outlined how we might get there. We crafted a philosophy for our organization around the idea we would aim to inform the change-makers, rather than advocating for any particular change ourselves:

    “We wanted all the people whose debates and actions wound up collectively producing education, through laws, policies, and practice, to do so with the maximum amount of information. They would proceed based on ideas and understandings anyway, whether right or wrong. As journalists, our job was to make sure those understandings were as accurate and evidence­-based as possible.”

    This approach led us to measure impact across two major categories: impacts where we ‘informed action,’ and impacts where we ‘informed debate.’

    It also helped us decide what not to measure. Anecdotally, at a recent small round table event for low income parents in Memphis, one family learned from Chalkbeat staff that free pre-kindergarten was available in their area; we love that one-to-one interaction and information, but something broader would be a more compelling impact in our system.

    The Chalkbeat Impact Taxonomy

    Under the heading of ‘Informed Action,’ we include things like lawsuits, policy changes, government investigations, and more — instances where real-world actions are taken as a result of our reporting.

    ‘Informed Debate’ covers various forms of citations, credits, links, pickups, mentions, and at times, republication of our stories in other news and information sources. In these cases, our stories are the drivers of conversation — often in public — about important issues in education.

    chalkbeat_impact_02

    chalkbeat_impact_01

    This taxonomy works for Chalkbeat; your mileage may vary.

    For a full menu of possible Impact categories, we recommend reviewing this Impact Glossary from The Center for Investigative Reporting.

    A Learning Organization

    We often refer to Chalkbeat as a “learning organization.” Yes, we cover education, and we pay special attention to data and research that can inform educational policy, but being a learning organization means that way try to pay as much attention to the evidence that reflects our own work: metrics and impact data.

    MORI sits near the core of that thinking, and the still-evolving status of tracking impact in journalism means we’re constantly learning from our own team’s use of the MORI platform.

    As a matter of fact, we’re had a quarterly all-hands meeting last week, and referring back to MORI has helped us find key impacts to highlight for the team. We’re also reporting on the quantitative data we pull from MORI to confirm, yes, we are on track to reach or surpass our impact goals for the year while growing audience. We don’t want to lean so heavily on stories with national mass appeal that we stop making impact at the local level.

    Examples of Chalkbeat's impact in MORI.

    Examples of Chalkbeat’s impact in MORI.

    We plan to keep refining MORI, and building the values of impact tracking into our planning and processes over time. By closely tracking the key impacts we have in our sphere of influence, we’re trying to learn how to plan for impact — to learn how to optimize our journalism to have the best chance to make a difference.

    Get Involved

    Read more about MORI, check it out on Github, and share what your impact taxonomy looks like, or what you think it might look like!

    Ryan Sholin is Chalkbeat’s Director of Product & Growth. He can be reached at [email protected] or @ryansholin on Twitter.

    Tagged: democracy education reporting grant reporting impact taxonomy media impact

    Comments are closed.

  • MediaShift received funding from the Bay Area Video Coalition (BAVC), which receives support from the Bill & Melinda Gates Foundation, to launch the MetricShift section to create a vibrant hub for those interested in media metrics, analytics and measuring deeper impact.

    About MetricShift

    MetricShift examines the ways we can use meaningful metrics in the digital age. We provide thoughtful, actionable content on metrics, analytics and measuring impact through original reporting, aggregation, and audience engagement and community.

    Executive Editor: Mark Glaser

    Metrics Editor: Jason Alcorn

    Associate Metrics Editor: Tim Cigelske

    Reader Advisory Board

    Chair: Anika Anand, Seattle Times Edu Lab

    Brian Boyer, NPR

    Clare Carr, Parse.ly

    Anjanette Delgado, Gannett

    Hannah Eaves, consultant, Gates Foundation

    Alexandra Kanik, Ohio Valley Resource

    Ian Gibbs, Guardian

    Lindsay Green-Barber, CIR/Reveal

    Celeste LeCompte, ProPublica

    Alisa Miller, PRI

    Connect with MetricShift

    Facebook group: Metrics & Impact

    Twitter: #MetricShift

    Email: jason [at] jasalc [dot] com

  • ADVERTISEMENT
  • ADVERTISEMENT
  • Who We Are

    MediaShift is the premier destination for insight and analysis at the intersection of media and technology. The MediaShift network includes MediaShift, EducationShift, MetricShift and Idea Lab, as well as workshops and weekend hackathons, email newsletters, a weekly podcast and a series of DigitalEd online trainings.

    About MediaShift »
    Contact us »
    Sponsor MediaShift »

    Follow us on Social Media

    @MediaShiftorg
    @Mediatwit
    @MediaShiftPod
    Facebook.com/MediaShift