A version of this guest post was originally published on the RJI blog.
Tracy Clark, a 2015-2016 RJI Fellow, believes newspapers with editor-selected email newsletters would have better engagement rates if the content were personalized to each user’s interest. She is in the midst of a pilot study with a large U.S. newspaper, which is simultaneously publishing two email newsletters: one includes editor-selected news content, the other features reader-selected stories. The personalized newsletters are based on Clark’s Reportory platform. This is a progress report.
I have always believed that content personalization is superior to non-personalized experiences. That’s why I’m thrilled that initial data from our customized news email pilot supports my hypothesis. Time will tell once the newness wears off on our service and we do the final analysis. Right now we are laser-focused on subscriber growth and continue to tweak our platform to ensure an optimized user experience.
Our experiment with a large daily U.S. newspaper has been in market for three months now. We are sending out two different newsletters simultaneously with the same amount of content, all aggregated from the newspaper and The Associated Press. In our version, users select the type of content they want to see. In the publisher version, an editor or journalist selects the content the reader will see.
We have reached 25 percent of their current subscriber base and so far the data leans toward supporting the personalized news approach. We will continue to grow the user base over the next three months to have six months of quantitative data on both services. We’ll also be deploying qualitative surveys to both readerships in order to get valuable satisfaction and preference feedback. All of this information will be published this fall as we unveil our publishing partner and the final results.
Readers Respond Positively to Personalization
Here’s a sneak peak of the performance so far. During the first month, our personalized news emails saw a higher open rate and substantially higher click-through rate (CTR) and click-to-open ratio (CTOR) than their editor-selected news emails. These numbers have stayed fairly similar month-to-month. We’ll release more comprehensive data at the conclusion of the study.
Let’s compare the performance of both newsletters against industry averages. IBM recently released its annual email benchmarking study that includes email performance data from 750 companies representing 3,000 brands in 40 countries. I also pulled recent stats from MailChimp, another email service provider. For reasons unknown, both of their average open rates and CTR and CTOR stats are well below both the publisher’s newsletter and our personalized newsletter. This bodes well for the publisher we are working with and extremely well for our hypothesis that personalization is superior:
Last year Digiday and others clamored over The New York Times’ 70 percent email open rate. While that statistic is phenomenal, they also mentioned that the Times has a 12-person newsletter staff. Many newsrooms across the country have greatly increased the number of newsletters they provide their readers (The Washington Post has more than 75 newsletters, according to Digiday). I petition that instead of deploying multiple niche newsletters that require manual editing and publishing, that media executives instead consider investing in a content personalization vehicle that automatically delivers personalized newsletters for each and every recipient.
Think about the numbers: Let’s say a dedicated newsletter editor at a medium-size publication makes $50,000 a year and he or she oversees three newsletters with a total of 40,000 subscribers. That means that the newspaper is paying this employee basically $1.25 per subscriber, not to mention the acquisition costs spent marketing the newsletters and providing ongoing email provider support. If instead the newspaper invested that $50,000 to building — or otherwise utilizing — an email personalization platform, then there wouldn’t be additional ongoing editor-related costs (albeit you can never forgo the technology and marketing costs). Please don’t think that I’m suggesting newspapers fire these newsletter editors. I’m suggesting that perhaps technology can automate their jobs, and their talents can be re-allocated to another part of the business that requires a human touch.
Unique Email Automation Challenges
Ultimately the experience depends on the content, and we’ve already had some snafus on the content based on the news feed that we have had to work through. For instance, the feed contains both English and Spanish articles but there’s no meta tag to distinguish language. Thus, we had to add code to detect a Spanish character and if it sees one, remove the article from the newsletter. This obviously isn’t fail-proof for articles on topics such as Beyoncé, for example.
Another “gotcha” that I’m dealing with is that if there are no articles that match a user’s preferences we revert to sending the latest articles from AP marked in the feed as “top news” so that we can always send five articles each day for every newsletter for research comparison purposes. However, for some reason in the feed AP flags all of their state lottery articles (and there are a lot of them) as top news, so that often times gets pulled into the newsletter. Someone in North Carolina, for example, could care less about results in the Ohio daily lottery, so I’m working now to re-code how articles get selected versus just relying on the “top news” meta tag.
I’m not the only one in the industry working on news personalization. Crain’s recently announced its own nationwide email news personalization initiative in 35 markets: “Crain’s new city brands feature smart email newsletters that customize business news based on location, reader preferences and engagement.” Moreover, according to Nieman Lab, The Washington Post is deploying “pop up” newsletters based on preferences, and they’re already seeing similar success to ours. “Click-through rates for the personalized newsletters are three times the average and the overall open rate is double that of the average for the Post’s newsletters,” writes Nieman Lab’s Ricardo Bilton.
I believe these two examples reinforce the notion of news personalization, and I’ll be following their efforts closely. More updates on our study to come this fall. Let me know if you have any questions in the meantime.
Tracy Clark is an RJI Fellow and founder of Reportory. As a nonresident fellow, she is conducting usability testing on her technology platform to create news digests for publishers’ subscribers based on individual preferences. Contact her at tracy@reportory.com.