The following piece is a guest post. Read more about MediaShift guest posts here.
In the news industry, it seems that the latest trend is to remove comment sections. Last year, Reuters, Re/code and Popular Science all got rid of them, and the Las Vegas Review-Journal kicked off 2015 by removing theirs (they have since turned them back on, after adding tools to help commenters self-police). In a Medium post explaining their original decision to turn off comments, the editors of the Las Vegas newspaper said, “The same platform that provides an opportunity for civil dialogue and an exchange of ideas also provides a platform for racism, bigotry and hatred. Those aren’t the types of conversations we want to host on our website.”
On the other end of the spectrum are the news organizations that are putting extra effort into their comment sections. Last week, Margaret Sullivan wrote that the New York Times would expand and build on their comment section. Community Editor Bassey Etim told Sullivan, “We’re lucky to have a celebrity class of commenters, and we want to elevate and recognize them in new ways.”
Comment sections provide news organizations with quite the conundrum. Comments are a way for readers to connect with one another, and with the journalists and editors reporting the news. But it is without question that comment sections can quickly turn uncivil, and news organizations often don’t have the resources or manpower to constantly moderate their comments.
At the Engaging News Project, we are dedicated to helping newsrooms meet their business and journalistic goals. One way we have done that is by researching comment sections, and finding solutions that we believe can help keep comment sections civil — and alive.
Here are a few of the research-based strategies that we believe can help newsrooms improve their comment sections.
Journalist Involvement
We began our research on comment sections by looking at what would happen if a journalist interacted with commenters. For our study, forthcoming in the Journal of Computer-Mediated Communication, we partnered with a local news station and tested our ideas on its Facebook page. Based on a schedule prepared by the research team, the station varied who interacted with commenters for 70 political news posts. For each post, one of three things happened: 1) a prominent political reporter interacted with commenters, 2) the station, using a generic station logo, interacted with commenters, or 3) no one interacted with commenters.
After analyzing nearly 2,500 comments for the presence of incivility, we concluded that the chances of seeing an uncivil comment were reduced when the reporter interacted with commenters (compared to having no one interact). The effect was not huge, but incivility was significantly dampened.
So what does this mean for newsrooms? It means that incivility in comment sections can be reduced by having a journalist say something in the comment sections. It doesn’t have to be much — just letting commenters know that a journalist is present can improve the tone of the comments.
We know journalists don’t have free time, but we found that a reporter was able to improve the tone of the discussion by commenting, on average, four or five times. Our finding also justifies other experiments looking to improve the tone of comment sections, such as inviting an expert to contribute to the space.
If a journalist wanted to get involved in the comment section, some suggestions for what to say include:
- Respond to questions
- Ask questions
- Provide additional information
- Encourage and highlight good discussion
Comment Structure
Most news comment sections are designed as one column of responses in a vertical manner. Yet several news organizations have been experimenting with new formats (for example, see The New York Times and The Washington Post). For our most recent project, we were curious to see what commenters would do when given a different comment structure.
In our study, we compared a one-column comment section to a three-column section that organized comments by whether they favored, opposed, or had questions/other comments about the legalization of marijuana.
We found that study participants rated a three-column comment section more favorably than the one-column comment section. Participants also were more likely to leave a comment in the three-column comment section.
There were limits, however, to using the three-column comment structure. There were some indications that study participants were less engaged with the third, right-most column on the webpage. They were less likely to press a “Like” button next to the comments in the right-most column compared to when the same comments were interspersed with the others in the one-column format.
There is still more research to be done on comment section structure, but our research shows that the structure of a comment section can affect commenting behavior. Figuring out new design strategies for presenting comments represents an important next step.
New Approaches to Comment Moderation
Photo by Michael Himbeault and used here with Creative Commons license.
An area in which we’re interested in doing more work is comment moderation. More specifically, we want to find new ways to moderate comment sections.
One of our research associates spent last summer attempting to code for incivility in online news comments in a variety of ways — using paid Workers in Amazon’s Mechanical Turk, using undergraduate students in the Mechanical Turk sandbox, and using machine learning software to automate the process. Relying on a previously coded data set as the comparison “Exemplar” coding, our researcher compared how well these groups and the program were at accurately detecting incivility.
Although we never reached perfect agreement, tweaking and editing our instructions consistently made us better. We are hoping to do more research in this area in the future.
We know that there is no perfect answer to fix comment sections. But we hope that our research can help newsrooms make their comment sections a better place for everyone.
Katie Steiner is the Communication Associate for the Engaging News Project, based at the Annette Strauss Institute for Civic Life at the University of Texas at Austin. Follow the Engaging News Project on Twitter at @engagingnews.