Last month, Jessica Clark and I explored how various Public Media 2.0 projects are measuring their level of success in informing and engaging publics. We found that many public media organizations are struggling to measure impact — and some are relying only on traditional indicators of reach, as opposed to other elements of impact such as relevance, inclusion, engagement or influence. Some projects, however, are taking a more holistic approach that is matched closely to their mission.
The international human rights group Witness, which provides training, support and visibility for local groups producing documentaries about human rights issues, has created a Performance Dashboard that tracks more than just the number of viewers. Using “at a glance” metrics, descriptive analysis and direct feedback from participants, the Performance Dashboard provides a concise overview of impact.
It combines traditional metrics — such as sales and licensing numbers, email subscriptions, blog statistics — with more nuanced data, including a timeline indicating progress of core partnerships. These reports are published twice per year on the Witness website, and they are made available to other organizations under a Creative Commons license.
Videos With a Purpose
Witness is able to efficiently track progress in large part because they begin each media project with clear advocacy goals. According to Sam Gregory, Witness’s program director, all work “springs out of an advocacy strategy.” He said Witness is focused on “making videos for a purpose as opposed to making videos about an issue.”
Each video project starts with the completion of a Video Action Plan, which encourages partners to think purposefully about intended impact, avenues for action, and measures of success.
Some of these measures of success are particularly striking. For example, Witness worked with the Centre for Minority Rights Development (CEMIRIDE), a human rights organization, to create a film about the displaced Endorois community in Kenya. The film ended up being presented as evidence in a landmark case in which the African Commission on Human and Peoples’ Rights ruled in favor of the Endorois community. Last month, the African Union, the highest legal authority in Africa, ruled in favor of the earlier decision and ordered the Kenyan government to provide the Endorois with compensation and reinstate their land.
While a direct causal link can be difficult to prove, clearly this film did its job. In a case such as this, the element of impact that is most important is influence, not reach. Gregory explained that even if only a few people saw the film, the film achieved its desired impact because they were the people with the power to decide the case.
New Focus: User-Generated Video
Witness hopes to broaden its impact with a new strategic vision that addresses the exponential growth in user-generated video. The organization is focusing on how user-generated video can be used by human rights advocates. (MediaShift reported on the organization’s earlier experiments with viral video in 2006.) Witness currently trains about 500 people in human rights filmmaking across the globe per year, and recognizes the need to shift to a more scalable training approach. One of the ways that Witness will make this shift is by developing shared virtual spaces for fostering discussion on what works and what doesn’t.
Yvette J. Alberdingk Thijm, Witness’s executive director, explained the strategy in a blog post:
Right now and right here Witness, with your help, can exponentially expand its impact. But the demand for our services is far greater than our capacity. Witness’s New Strategic Vision is designed to scale our impact. So beginning in 2010, in addition to continuing to train and support individual grassroots organizations, Witness will forge relationships among organizations and networks, creating a broader, more interconnected global human rights community. By doing this, we’ll play a seminal role in forging coalitions that seek shared goals, with video emerging as the common language across all types of borders. In addition, we will scale our work by creating video toolkits and other web tools that facilitate knowledge sharing.
With the new focus on networked campaigns, in some ways, impact will become more difficult for Witness to track. What is the most effective way to measure impact when the media in question spans across so many different modes, timeframes, countries and (sometimes overlapping) networks?
In the future, Witness will likely spend more energy tracking the connections that form within and among networks. The Witness team is currently working through the process of adding new categories to the current Performance Dashboard.
The dashboard offers a great model for other media projects. But it’s also clear that projects without similar, specific advocacy goals will likely have a harder time making use of the tool. Outlets and creators with more neutral goals of spurring discussion or raising awareness may have to turn to some of the existing impact assessment toolkits — or perhaps even develop their own.
Katie Donnelly is Associate Research Director at the Center for Social Media at American University where she blogs about the future of public media. With a background in media literacy education, Katie previously worked as a Research Associate at Temple University’s Media Education Lab in Philadelphia. When she’s not researching media, Katie spends her time working in the environmental field and blogging about food.