This guest post by Talia Stroud and Shannon McGregor was originally published on the Engaging News Project site.
Read the full series on Measuring Impact
Solutions journalism is reporting that focuses on responses to social problems, instead of just the social ills themselves. It investigates how people, institutions, and communities work towards solutions, and whether and why solutions work.
Newsrooms across the country have implemented reporting with a solutions journalism focus, but little is known about how the public and communities respond. In this report, we ask an overarching question: Can a solutions journalism campaign change public discourse? The research described in this report was jointly funded by the Solutions Journalism Network and the Engaging News Project (with grants from the Democracy Fund, Hewlett Foundation, and Rita Allen Foundation).
On March 6, 2016, the Chattanooga Times Free Press published a series focused on poverty called “The Poverty Puzzle.” In this report, we track whether the series changed public discourse by looking at Twitter and public government documents.
Findings suggest that:
- The public tweeted more about poverty in the days following the publication of the series.
- Community churches and opinion leaders did not use more poverty-related terms in their tweets following the series.
- In the Chattanooga City Council meeting minutes, there were 4 mentions of poverty-related terms in the 12 meetings prior to the series, compared to 26 mentions in the 12 meetings following the series.
For this research, we chose to use Twitter as our method of monitoring online discourse as many user accounts are accessible to the public and not set to private, as can be the case on Facebook. Pew Research reports that only 16% of U.S. adults use Twitter and even fewer – 9% – get news from Twitter. Thus our analysis captures only a small fraction of the possible ways in which people could respond to the series. Nonetheless, the ability to examine unprompted responses from Twitter users within a geographic area is a considerable advantage.
To do this research, we identified the most common words in the Times Free Press series related to poverty. We then analyzed how frequently these terms appeared in tweets from Chattanooga and in Chattanooga City Council meeting minutes. We collected 5,393,325 tweets from Chattanooga, Tenn., from December 1, 2015, to June 1, 2016, using the program Crimson Hexagon. Of these, 4,768 contained poverty-related terms derived from the text in the series. We looked at public tweets that were geotagged in Chattanooga, Tenn., and at tweets from community opinion leaders (including community churches) identified by the Times Free Press. We also gathered the minutes from City Council meetings during the same period. More details about our method can be found at the close of the original report.
Public Tweets about poverty increased in days after series published
Our analysis shows that tweets about poverty in the community increased following the publication of “The Poverty Puzzle” series by the Chattanooga Times Free Press. Comparing the week before the series to the week afterward, tweets on the topic rose by 27.9 percent. However, the uptick did not extend beyond several days.
Below is a graph of poverty-related tweets from the public by day. Looking at the 7-day moving average (the darker line in the graph), there is a notable increase in poverty-related tweets following the launch of the series on March 6, and another uptick in poverty-related tweets on March 31.
We statistically assessed the change in Twitter-based poverty discourse following the publication of the Times Free Press’ series. The three days directly following the series’ publication show a significant increase in poverty-related tweets, but this trend does not continue beyond the first few days.
We looked at the tweets to get a better sense of the conversation. Some tweets linked directly to the story in the Times Free Press. Others discussed poverty in Chattanooga. For example, this tweet “I’ve been homeless three times. The problem isn’t drugs or mental illness — it’s poverty” linked to another story about poverty. Another subset of tweets in the week following the series’ publication shared opinions about revelations from the series, such as: “Paying school cafeteria employees more than fast food restaurants would doesn’t mean we pay them enough. $8.43 isn’t a living wage.”
Paying school cafeteria employees more than fast food restaurants would doesn’t mean we pay them enough. $8.43 isn’t a living wage.
— Nick Wilkins (@NickkWilkins) March 10, 2016
We also observed another increase in poverty-related tweets from citizens in Chattanooga four weeks post-publication. The uptick in the fourth week, most noticeable on March 31, seems to be tied to widely tweeted news that a Chattanooga waitress filed a minimum-wage class action lawsuit against Ruby Tuesday.
Discussion of poverty in City Council meetings increased after series
There was an uptick in poverty-related words used in Chattanooga City Council meetings after the series was published.
We gathered minutes from city council meetings before and after the publication of the series – 24 meetings took place in this period. In the 12 meetings before the series was published, the 11 poverty-related words appeared only four times. In contrast, in the 12 weeks following the publication of the series, those same words appeared 26 times. The graph below displays the sharp upticks following the March 6 publication of “The Poverty Puzzle” in the Times Free Press.
We examined the meeting minutes for several of the peaks to understand the qualitative nature of the discussion. The March 22nd and May 17th meetings included a report from the Youth and Family Development divisions, which discussed services for those with low incomes. On May 5th, a local resident addressed the council about the issue of poverty. There also was a special presentation that referenced unemployment. The series was not directly mentioned, and these data cannot tell us whether the series is responsible for the spikes.
Opinion leaders did not drive poverty discussion on Twitter
Our analysis of tweets from opinion leaders, identified in conjunction with local reporters, did not show the same responsiveness to the Times Free Press poverty series as did the public tweets.
A graph of poverty-related tweets from the opinion leaders across time can be found below. There was no notable increase in poverty-related tweets following the launch of the series on March 6. Of the 63 accounts we identified as opinion leaders, only 23 tweeted in the six months during which we collected data. Those 23 accounts averaged slightly more than 11 tweets per account over the time period. At least in this instance, Twitter may not be the place where opinion leaders engage.
The only spike we observe comes some nine and a half weeks after the series was published. We examined the tweets in this spike, which lasts from May 12 to May 19, and found a flurry of tweets about an upcoming church conference about poverty called “Re-Think Poverty.” It appears that the publicity about this upcoming event drove an increase in poverty-related tweets from opinion leaders. As will be discussed shortly, opinion leaders affiliated with churches were responsible for this uptick.
Tweets related to the Re-Think Poverty conference included: “Good intentions aren’t enough–gather your ministry networks at Rethink Poverty & learn how to help without hurting” and “How is your church connecting people to institutions and services that can help them escape poverty?”
As we did with the public, we assessed whether there was any statistically meaningful increase in poverty-related Tweets by opinion leaders following the Times Free Press’ poverty series. This analysis, described in detail in the method section at the close of this report, reveals a significant mid-May uptick, but little else.
Churches drove modest uptick due to event, not coverage
In addition to the 16 churches listed as opinion leaders by the Times Free Press, we looked at an additional 24 churches that call Chattanooga home. We found no tweets matching our poverty-dictionary terms from these additional churches. However, we should note that the May uptick in poverty tweets among the opinion leaders (detailed in the section above) does stem from churches. This increase can be attributed to three accounts tweeting about the upcoming Re-Think Poverty conference.
Conclusion
This study provides some support for the ability of long-form solutions journalism series, like the Chattanooga Times Free Press’ “The Poverty Puzzle,” to affect public discussion, as measured through social media.
There are some limitations to this content analysis – for example, as most Facebook accounts are private, we were only able to examine public Twitter accounts for social media discussion about poverty. Further, we note that this study cannot prove causal relationships – it is possible that something else happened at the same time as the Times Free Press series that is responsible for the patterns we document. Inspection of the tweets, however, makes this possibility unlikely. Finally, it isn’t possible to know whether a certain component of the series, such as its length or its use of solutions journalism, might be responsible for the observed patterns. It also could be something specific to the issue of poverty. Indeed, research shows that tweets follow a different pattern than online news in response to major news events like elections.
The share of poverty-related tweets in Chattanooga rose significantly following the series publication. The spike, however, lasted only for a few days. This suggests that while impactful, the attention span of social media to the issue was limited. Yet social media in general, and Twitter specifically, may display short-lived attention to issues as compared to other media or measures of public opinion. We also uncovered that the public led the news-driven conversation about poverty without the influence of opinion leaders. Finally, the spike in poverty-related word usage in the City Council meeting minutes suggests that the series may have influenced government dialogue.
We followed up with Joan McClane, author of “The Poverty Puzzle,” about the impact of the series beyond Twitter. “The community response to The Poverty Puzzle has been overwhelming,” she told us. “Many organizations and citizens have reached out to us to get involved in ways ranging from mentoring to using the series in their classrooms to holding forums to funding innovation regarding poverty.” She said she also found the response to be compassionate and respectful across political divides. Her experience emphasizes that future research should look at how solutions journalism sparks community dialogue and reactions beyond social media.
The methodology and statistical analysis can be found on the Engaging News Project site.
Talia Stroud is the Director of the Engaging News Project at the Annette Strauss Institute for Civic Life and Associate Professor of Communication Studies at the University of Texas at Austin.
Shannon C. McGregor is a doctoral student and assistant instructor in the School of Journalism at the University of Texas at Austin.
