In 2014, Chalkbeat, where I work, developed and started using a WordPress plugin for tracking impact. We called it MORI — Measures of Our Reporting’s Influence. MORI grew out of one of our key beliefs: Journalists can make a difference, but it’s the ability to measure that difference that multiplies our impact over time. If we can document how, why, when and where we made a difference, we are more likely to repeat our success.
The latest version of the MORI Impact Tracker is now available as an open source release, a WordPress plugin free to everyone to download and install.
For those who do try out MORI, you’ll quickly be faced with questions that every organization focused on tracking impact has to answer: What counts as journalistic impact? How should we categorize and classify different types of impact? What type of impacts are most important to track? Can we track everything? Should we?
A Theory of Change
The answers to those questions depend heavily on your organization’s “Theory of Change.”
For a nonprofit news organization, a Theory of Change explains how you’re going to get from step 1 — journalism — to step 3 — impact. Are you going to explicitly seek to expose wrongdoing and corruption in government, assuming that by bringing malfeasance to light you can generate a response that shuts bad behavior down? Or maybe you want to draw attention to voices without access to power, in the hopes that giving a platform to marginalized people is a way of empowering them? There are many possible paths to impact.
When Chalkbeat launched MORI in 2014, our white paper outlined how we might get there. We crafted a philosophy for our organization around the idea we would aim to inform the change-makers, rather than advocating for any particular change ourselves:
“We wanted all the people whose debates and actions wound up collectively producing education, through laws, policies, and practice, to do so with the maximum amount of information. They would proceed based on ideas and understandings anyway, whether right or wrong. As journalists, our job was to make sure those understandings were as accurate and evidence-based as possible.”
This approach led us to measure impact across two major categories: impacts where we ‘informed action,’ and impacts where we ‘informed debate.’
It also helped us decide what not to measure. Anecdotally, at a recent small round table event for low income parents in Memphis, one family learned from Chalkbeat staff that free pre-kindergarten was available in their area; we love that one-to-one interaction and information, but something broader would be a more compelling impact in our system.
The Chalkbeat Impact Taxonomy
Under the heading of ‘Informed Action,’ we include things like lawsuits, policy changes, government investigations, and more — instances where real-world actions are taken as a result of our reporting.
‘Informed Debate’ covers various forms of citations, credits, links, pickups, mentions, and at times, republication of our stories in other news and information sources. In these cases, our stories are the drivers of conversation — often in public — about important issues in education.
This taxonomy works for Chalkbeat; your mileage may vary.
For a full menu of possible Impact categories, we recommend reviewing this Impact Glossary from The Center for Investigative Reporting.
A Learning Organization
We often refer to Chalkbeat as a “learning organization.” Yes, we cover education, and we pay special attention to data and research that can inform educational policy, but being a learning organization means that way try to pay as much attention to the evidence that reflects our own work: metrics and impact data.
MORI sits near the core of that thinking, and the still-evolving status of tracking impact in journalism means we’re constantly learning from our own team’s use of the MORI platform.
As a matter of fact, we’re had a quarterly all-hands meeting last week, and referring back to MORI has helped us find key impacts to highlight for the team. We’re also reporting on the quantitative data we pull from MORI to confirm, yes, we are on track to reach or surpass our impact goals for the year while growing audience. We don’t want to lean so heavily on stories with national mass appeal that we stop making impact at the local level.
Examples of Chalkbeat’s impact in MORI.
We plan to keep refining MORI, and building the values of impact tracking into our planning and processes over time. By closely tracking the key impacts we have in our sphere of influence, we’re trying to learn how to plan for impact — to learn how to optimize our journalism to have the best chance to make a difference.
Get Involved
Read more about MORI, check it out on Github, and share what your impact taxonomy looks like, or what you think it might look like!
Ryan Sholin is Chalkbeat’s Director of Product & Growth. He can be reached at rsholin@chalkbeat.org or @ryansholin on Twitter.