There are thousands of meaningful ways to create positive change with storytelling, as well as metrics to measure the success of those changes. With all of these possibilities, where do you even start thinking about measuring impact? This was the problem we had in mind when we created StoryPilot, an online platform that tracks the creation and impact of over 500 social issue documentaries. We used a few simple questions to organize information and provide a starting place for thinking about the difference that a film can make in the world:
- What issue is the film addressing?
- How was the film made?
- Who was involved?
- What changes were we able to observe?
Knowing that every film’s story is different, we gathered publicly available data that speaks to broadly applicable types of impact. Here are three things you can look for on StoryPilot and how to interpret what the data is telling you.
One of many influence indicators produced by the StoryPilot system.
Gaining Attention: Reach, Network Building, and Amplification
Tracking reach for the sake of reach is often criticized as measuring “vanity metrics.” However, there are a number of ways to do this, whether through reaching a viewership or building visibility within a social network. Within StoryPilot, we try to capture several ways of measuring a film’s potential for influencing a wide audience. By measuring trailer views, known to be a reliable proxy for film viewership, we gain a sense of the reach of the film itself. We also look at network building by measuring social media following and engagement. Tracking press coverage gives us a sense of how much the message of the film has been amplified by other media platforms. These metrics aren’t enough to definitively claim social impact, but they show us a foundation upon which more meaningful types of change can be built.
Information seeking
What is the first thing you do when you get curious about a new movie? Searching for information online is pretty close to the top of the list. Online searching is one form of “information seeking behavior,” a type of impact that demonstrates curiosity about a topic. To measure information seeking, we look to Wikipedia pageviews.
StoryPilot information seeking graphic.
While Wikipedia is far from the only place to find information, its peaks and valleys are indicative of larger patterns in interest. However, a special thing about the online encyclopedia is its ability to inspire tangential learning, where interest in a film leads to learning about broader topics.
For example, in February 2013, “How to Survive a Plague” was nominated for an Academy Award. Predictably, we saw a spike in the number of people visiting the documentary’s Wikipedia page. When we looked just a bit further, we found a simultaneous spike in the number of people visiting the page for ACT UP, an activist group that was a focal point within the film. The documentary provided a critical link between the issues in the film and previously unreached audiences.
To investigate issue-related pages on your own, use the Wikipedia Pageview Tool.
Political Mentions
For some goals, it doesn’t matter how many people you reach, but who you reach. One prime example of this is in political change, where the support of politicians who will champion your cause is critical to success. A film that took this approach was “The Invisible War” (2012), which held targeted screenings to gain the support of influential political actors. We found that the film was brought up five times in Congress and informed changes to the National Authorization Act for Fiscal Year 2013. Within StoryPilot, you can find out which Congressional representatives mentioned the film, what states they represent, and which other terms they used.
You can explore some of the data using the Sunlight Foundation Capitol Words tool.
Looking Forward
We’re continuing to add a wider variety of data that represents more types of impact. Check out http://StoryPilot.org to see these metrics for a variety of films around 16 social issues. along with information on festival attendance, cast and crew, partners and more. For more information on choosing metrics for your film or social issue campaign, take a read through our Impact Playbook.
Lauren Hanson is a Research Analyst at the Harmony Institute, an organization that conducts research to understand how media works. Read more about Harmony Institute and their work in the field here. Lauren works across the Harmony Institute’s research process, from designing and conducting independent and collaborative studies to communicating insights to the research community, industry partners, and storytellers. She has a special interest in interactive media, transmedia, and games.
