This post was written and originally published by CIR and is republished here with permission.
Some stories change laws. Others force governments and companies to change their policies. Advocacy and interest groups often use these investigations in their own work. Still other stories change a community’s awareness of an issue.
The first three examples of impact are real, discrete events we can measure. The last one – changing awareness of an issue – proves to be much more difficult to know for sure. How can a media organization really prove a project increased the public’s knowledge? And does this even matter for lasting social, political and cultural change?
I wanted to measure how a piece of investigative reporting changed public awareness on an issue. So I designed a survey that reached residents both before and after the story published, then analyzed the data for changes in attitude. Here’s a rundown of what we did and what we learned.
In early 2014, reporters from The Center for Investigative Reporting were working on an investigation of pesticide use in California. As they engaged in on-the-ground reporting in strawberry-growing communities, it became clear that residents they were speaking with had little idea about the health risks they faced.
When we considered the potential impact of the investigation, I hypothesized that there were few actors within the state who had an incentive to act upon the information we would be making public. The reason: The close ties between the well-resourced chemical companies and the California Department of Pesticide Regulation made the potential for real change unlikely, at least in the short term.
So I asked myself: Who does have an incentive to act? The answer: residents in the most affected communities. Thus, we identified a specific impact goal for the Dark Side of the Strawberry as an increase in awareness in the communities that are most affected by pesticide use. To achieve this goal, we knew we’d need a specialized plan for distribution and engagement. And we needed to set some baselines about current levels of awareness of pesticide use in the strawberry industry.
There were no data about levels of knowledge of pesticide use and associated health risks in strawberry-growing communities, so I designed a survey research project to gauge this in one community with a thriving strawberry industry: Oxnard, California. The survey project was made possible with funding and student effort from the University of Southern California.
The pre-project survey, distributed to a statistically significant random sample of residents in Oxnard, confirmed that residents were not overly concerned with pesticide use and potential health effects. The vast majority of respondents (92 percent) never had reported concerns to anyone, and 75 percent believed pesticides had not affected their health or were unsure.
To inform our distribution and engagement strategy, we also asked respondents how they typically get news and information about their local community. Respondents cited TV, the Ventura County Star newspaper and radio and online news as top sources. You can see the methodology and detailed response information in the full case study.
What we learned
We waited six months after our last on-the-ground engagement in Oxnard before distributing post-project surveys to a random sample of residences.
A simple comparison of pre- and post-project survey results showed some notable shifts in responses. First, respondents reported higher levels of concern about being exposed to pesticides while at home or work.
This increase was even more pronounced among Spanish-dominant respondents. When we analyzed responses based upon the language in which surveys were completed (Spanish or English), the pre-project survey showed a slight negative correlation between language and level of concern about pesticide exposure. Spanish speakers were less likely to be concerned than were English speakers. In the post-project survey, there was a significant shift, and Spanish speakers became more likely to be very concerned about pesticide use than English speakers. In fact, in the post-project survey, not a single Spanish-speaking respondent reported that he or she was “not at all concerned” with pesticide exposure.
While there was a slight decrease in the proportion of respondents who reported that they had heard about pesticides in the past year (92 percent in pre-project survey and 86 percent in post-project survey), the percentage of respondents who indicated that they had heard about pesticides in the media increased. In the pre-project survey, 41 percent said they had heard about pesticides from the media in the last year; this percentage grew to 43 percent in the post-project surveys.
Of respondents who reported that they had heard about pesticides from the media, 35 percent reported this information had come via television, and 32 percent selected the Ventura County Star.
Future research and conclusions
From this project, there is one lingering question ripe for future analysis: While CIR translated the project into Spanish, we did not locally distribute this content with a Spanish-language partner. And when I conduct a scan of Spanish-language media, there are no stories that directly reference CIR’s investigation or fumigant use in the strawberry industry in Oxnard. Nonetheless, we still saw the greatest shift in levels of concern among the community’s Spanish speakers. Future research could focus on answering the question: Under what conditions do English investigative projects reach Spanish-speaking audiences?
The story likely was picked up by Spanish-language radio and/or TV. And it seems possible that the investigation resonated among dominant Spanish speakers, many of whom work in agriculture, because the project’s content was framed in a way that was not attacking the strawberry industry as a whole, so it did not directly threaten the livelihoods of agricultural laborers. Instead, the story focused on the history of the industry and the failure of government agencies to enforce regulations and use science to inform decision-making. However, because we do not know where the story was covered in Spanish-language media, if at all, we cannot be sure.
There are three major takeaways from this analysis, all of which are relevant for future investigative projects:
1) Engaging with an established local stakeholder organization early was key to subsequent structural/policy change at the local level.
On-the-ground reporting allowed CIR reporters to build relationships with teachers who were part of the local teachers union.
Stakeholder groups that have an incentive to act might not be obvious. In this case, agricultural labor organizations, while representing workers whose health could be at risk, would not speak with reporters. But because the farm fields often ran alongside schools, pesticides also became an education issue. Ultimately, the story focused on schools, and a powerful group of players who might not have been immediately obvious – teachers – became active in pushing for change.
2) Impact distribution – in which we take a national story and tailor it with local distribution and engagement – resulted in increased awareness of and concern about pesticide use in Oxnard
Local media – especially the Ventura County Star – have credibility and a pre-existing relationship with the community. They proved to be important partners for reaching affected communities. This is a finding that has been replicated in each of our impact analyses and has been reinforced anecdotally through other CIR projects, from peer media organizations and through academic and other research.
While it’s clear that local distribution and engagement were keys to catalyzing some on-the-ground change, state officials were unable to ignore the story completely thanks to this project’s broad reach. State regulators have curtailed some pesticide use in the wake of the reporting and begun to lay the groundwork for changing rules for pesticide use around schools and homes. And following the local newspaper’s story, the Ventura County Board of Supervisors also increased its pesticide monitoring.
3) CIR’s continuous reporting on this story fed the long tail of impact
CIR reporters and editors stuck with this story, staying abreast of local developments, meetings and calls for government investigation and engaging with the public around this topic. By continuing to shine a light on this issue and providing relevant information consistently, CIR helped to keep this issue on the public – and political – agenda.
More Reading
You learn more about the methodology and response information in the full case study.
Lindsay Green-Barber is the Director of Strategic Research at The Center for Investigative Reporting. CIR is one of the largest nonprofit newsrooms dedicated to investigative reporting. It recently launched a weekly national public radio show and podcast called Reveal, produced with PRX. Learn more at revealnews.org.
