A version of this article was originally published by the Donald W. Reynolds Journalism Institute.
In a head-to-head experiment, e-newsletter content chosen by the reader outperformed e-newsletter content selected by an editor.
Over the past year as an RJI nonresidential fellow, I partnered with the Austin American-Statesman to set up a study to compare one of their editor-curated newsletters, Midday Break, with Reportory, a news personalization technology I developed. First, I white-labeled my platform to look exactly like Austin’s interface and the Midday Break email. We called this rebranded personalization platform Austin American-Statesman’s News for You.
Then I worked with their data team to ensure we could personalize the content based on existing tags and data scraping techniques. Readers were able to personalize content by sources (Austin and/or The Associated Press), categories (such as news, sports and technology) and/or keywords. We had hundreds of keywords for readers to select from, and they were able to insert their own.
Using data science to pick stories
The algorithms that power Reportory take all of the user’s selections and match it against all content within the database, first prioritizing strong matches across all three (source, category, keyword), then continuing to make further matches based on two indicators (source and keyword, category and keyword, or source and category), followed by only one match until the desired number of articles was met.
We chose to deliver five customized articles daily for this experiment. Once we ensured the personalization was working, we deployed the technology to Austin readers and implemented both organic and paid marketing campaigns on Facebook to secure 250 registered users. We kept both platforms running for six months, delivering emails at noon each weekday. Midday Break delivered the same editor-selected stories for all users. News for You provided a unique set of articles each day to each user based on the preferences they indicated.
Examples of the test newsletters for readers interested in sports and politics
Which newsletter performed better?
Aggregate engagement data below show that News for You outperformed the Midday Break on all variables except unsubscribe rate during the test period. Daily emails from News for You were nearly twice as likely to be opened and almost three times more likely to have a story actually read by a subscriber.
At the conclusion of the study we conducted a survey to both distribution lists and found that participants from News for You were slightly more satisfied with service quality than Midday Break subscribers. In terms of recommending the service to a friend, News for You subscribers were moderately more likely to recommend than Midday Break subscribers. The competitive advantage of News for You is notable in that Midday Break has a longer life span and significantly more subscribers.
This study offers some evidence for the debate between the trustee and the market models of journalism. In the trustee model, journalists and editors select news content based upon professional standards and judgment. The market model dictates that news creation and selection should be audience-driven. It is uncommon for a major city newspaper to agree to share the results of a field experiment that in essence tests the debate over whether increased personalization of news content results in more satisfied customers.
Ample literature suggests that traditional newspapers must constantly adapt to market conditions in order to remain in business. The digitization trend for news products and sharing through social media are realities that empower the news consumer.
An opportunity for daily metro papers
While this study was limited in scope and scale, it does provide some evidence that news consumers prefer greater control of their content. This experiment supports the contention that the market model of journalism is the path to economic survival for traditional city newspapers.
During my research I spoke with many in the industry who are extremely concerned about giving up editorial power and enabling a reader to funnel their news into the narrow, biased filter that aligns to their personal views. However, I argue that individuals are already doing this in their social networks by the friends they choose, the companies they follow and the news they share, thus why wouldn’t the news industry better support (and capitalize on) this market demand for a more personalized and relevant news consumption experience?
I have always disagreed with the notion that all print readers receive every section of the news regardless of what they actually read. What a waste. Instead, let’s focus on delivering exceptional journalism and empowering the reader with tools to uncover and digest the news they want and in the way they want (be it web interface, mobile app, email newsletter). In the end we are creating a more symbiotic relationship between the media and the reader versus the current dichotomy of news “preaching” down to the masses about what a few editors deem society should care about.
——–
We are extremely appreciative of our publishing partner Austin American-Statesman and especially Steve Dorsey, vice president of innovation and planning, for teaming with us on this initiative. It is the willingness on behalf of busy executives such as Steve to volunteer his time to studies like this that truly help advance our field.
Tracy Clark is an RJI Fellow and founder of Reportory. As a nonresident fellow, she is conducting usability testing on her technology platform to create news digests for publishers’ subscribers based on individual preferences. Contact her at [email protected].
