• ADVERTISEMENT

    Analyzing NPR’s Facebook Publishing Times

    by Dan Frohlich
    March 25, 2016

    This post was originally published on the NPR tumblr.

    A while back, we ran an analysis on our posts to Facebook and concluded that the platform was largely time independent in terms of whether a post succeeded. A post at 4 am on Saturday should be just as likely to find an audience as one at 10 am on Tuesday. Despite a solid statistical analysis, our editors found this data hard to swallow. So Dan Frohlich, our metrics analyst for digital, circled back to this question for us. His findings are below. They’re inconclusive but do seem to point us in a new direction. I’ll follow up with my own thoughts after his analysis. – Wright Bryan

    Our audience engagement team has routinely had questions about whether publish time on Facebook impacts the performance of a story. Up until now, we’ve relied a lot on the graph of when our fans are online in Facebook Insights – something we refer to as the “Facebook whale.” But because we know our users see older content in their feed hours or even days after it was originally posted, we wondered, does it actually matter if something is published during a time when fewer Facebook users are online? That’s the question we set out to answer.

    ADVERTISEMENT

    I want to stress upfront that this is not an experiment, it’s simply observational data based on posts we published over a 6-month period. I’ll explain more on why that matters later on.

    Firsts Things First, When Do We Publish Content on Facebook?

    I looked at the performance of 4,085 NPR link posts (excluding station links, video posts, etc.) from August 2015 through the first week of February 2016. The first thing I noticed is that we already publish most of our Facebook content during the daylight hours, peaking around noon (see graph below). This is also very similar across weekday and weekend. All times in this note are Eastern.

    Hour-By-Hour, Which Hours Perform Best?

    First I wanted to see how each hour of the day compares to the overall average of the 4,000+ posts. I looked at 4 different metrics: total reach (the number of people who saw a post), the total number of impressions (how many times a post was seen, including multiple views by one person), the number of link clicks (people clicking on a link back to npr.org), and the number of post consumers (people who interacted with the post in any direct way). The following table summarizes the results. A “No” means there wasn’t a statistically significant difference (at 95 percent confidence) between the overall average and how posts from a particular hour performed. All of the other time periods are marked on whether they were significantly higher or lower than the overall average.

    ADVERTISEMENT

    Only one hour (7 am) performed significantly above the overall average across all 4 metrics (meaning those posts had more link clicks, reached more people, and received more impressions/consumers). 10 am led to posts receiving significantly more people and garnering more impressions. 5 pm led only to significantly more impressions. Posts published at 3 am reached fewer people and had fewer impressions and posts at 8 pm had significantly lower scores on reach, impressions and consumers.

    This didn’t definitively answer the question of posting things on peak vs. off-peak hours. So …

    Comparing 7pm-6am to 7am-6pm

    For the next part of the analysis I compared posts published between 7pm-6am (off-peak) to posts published between 7am-6pm (peak). I found significant differences across all four metrics. Posts published during peak hours (7am-6pm) reached more people and received more impressions/link clicks/consumers per post. The table below shows the averages for the combination of each metric & time period.

    However, since this wasn’t a true experiment, there are some things we should consider first before we declare peak-hour posting to be more valuable.

    What This Data Does & Doesn’t Tell Us

    One major complicating factor is how our editorial decision making has been impacted by the belief that publishing time does matter. For example, if we approach Facebook thinking we should maximize the traffic to our “best” content, and we believe one of the ways we do this is by publishing that content during peak hours, then we’re effectively creating a self-fulfilling prophecy.

    • Peak hours => publish “best” content => best results
    • Off-peak hours => publish everything else => worse results

    So naturally it wouldn’t be a surprise that the hours that perform best in an analysis like this are the peak-traffic hours (if you assume that the quality of the content is more important than publish time). A true experiment would require us to roll a die or flip a coin to determine when we publish content (randomizing when our “best” content gets published). That would help control for our editorial judgment coming into play and we could more definitively answer this question. (We do not plan on actually running this experiment.)

    There’s also the question of breaking news. My gut tells me that U.S. breaking news is more likely to happen during peak hours, simply because that’s usually when people are working (for example, a politician is less likely to make an attention grabbing statement at 3 am than 9 am).

    Open Questions 

    • Are we manufacturing this result by publishing our “best content” during peak hours?
    • Even if peak hours (7am – 6pm) are better for publishing Facebook posts, do we want to publish more during periods that already include more posts?
    • What metric are we trying to optimize for? Historically we’ve put less weight on reach and more weight on consumers and link clicks. This data suggests that publish time may impact some metrics more than others.

    –Dan Frohlich

    The message I take from this analysis is that we should post more often from midnight to 8 am. Publishing our best material in the heart of the East Coast business day is either helping us or not hurting us. But neglecting the wee hours of the clock is a missed opportunity for stories that deserve an audience but are losing out in the competition for daytime slots. We are still reaching lots of people when we post after midnight. So my recommendation will be to add a few more posts so that we’re publishing into the feed at maximum intervals of 60 minutes.

    What do you think? Do you have any insights to share on posting to FB? Hit me up on Twitter (ha!) if you do. – Wright Bryan

    npr-logo

    Wright Bryan is NPR’s senior editor for engagement. Dan Frohlich is a digital metrics analyst in NPR’s Audience Insights group.

    Tagged: analytics breaking news facebook impressions npr performance scheduling
    • Soul Shaolin

      Thanks for sharing this analysis and all this data. The Facebook publishing times is really a recurring issue, I had to work on this many times and there is no perfect answers. Here is a few remarks and thought :

      – Besides the obstacles mentioned in this article I would add the difficulty to access real time data with Facebook. Since we can’t track metrics hourly it’s really hard to know when content are really distributed. You can breakdown your metrics hourly at a page level but you can’t at a post level. I tried to bypass this with GA by using traffic segmentation acquired from Facebook and by splitting Paid and Organic traffic. Looking at Paid traffic split by hours, It helped me to understand that Facebook is perfectly capable of delivering a constant flow of traffic all day long

      – I like to build my publishing schedule based on ER or CTR. Most of the time I rank all the posts I can by ER or CTR then I build a top and a flop post. Then I just count how many of them by hours to build my calendar. Like this I can easily avoid the risk of building my recommendation based on a few viral posts ruining my reach or impressions data.

      – But most of all, this analysis make me think in “Editorial Data” (can’t remember where I can read about this) . Instead of dealing with last 6 months data we should split our content by audiences target and categories. Since NPR have multiple Facebook pages It would be interesting to have some feedback on this. And because NPR music and NPR politics certainly not have the same engagement timing it would be interesting to look at, for exemple, age range and hour engagement.

      Without mentioning Instant Articles, AMP or Apple news, all this show us how much publishers and content producers are dependent of platform and how data is the key in this war. At the time of distributed content analysing data is more viral than ever.

      Thanks !

  • MediaShift received funding from the Bay Area Video Coalition (BAVC), which receives support from the Bill & Melinda Gates Foundation, to launch the MetricShift section to create a vibrant hub for those interested in media metrics, analytics and measuring deeper impact.

    About MetricShift

    MetricShift examines the ways we can use meaningful metrics in the digital age. We provide thoughtful, actionable content on metrics, analytics and measuring impact through original reporting, aggregation, and audience engagement and community.

    Executive Editor: Mark Glaser

    Metrics Editor: Jason Alcorn

    Associate Metrics Editor: Tim Cigelske

    Reader Advisory Board

    Chair: Anika Anand, Seattle Times Edu Lab

    Brian Boyer, NPR

    Clare Carr, Parse.ly

    Anjanette Delgado, Gannett

    Hannah Eaves, consultant, Gates Foundation

    Alexandra Kanik, Ohio Valley Resource

    Ian Gibbs, Guardian

    Lindsay Green-Barber, CIR/Reveal

    Celeste LeCompte, ProPublica

    Alisa Miller, PRI

    Connect with MetricShift

    Facebook group: Metrics & Impact

    Twitter: #MetricShift

    Email: jason [at] jasalc [dot] com

  • ADVERTISEMENT
  • ADVERTISEMENT
  • Who We Are

    MediaShift is the premier destination for insight and analysis at the intersection of media and technology. The MediaShift network includes MediaShift, EducationShift, MetricShift and Idea Lab, as well as workshops and weekend hackathons, email newsletters, a weekly podcast and a series of DigitalEd online trainings.

    About MediaShift »
    Contact us »
    Sponsor MediaShift »

    Follow us on Social Media

    @MediaShiftorg
    @Mediatwit
    @MediaShiftPod
    Facebook.com/MediaShift