On July 5, 1989, a pilot episode of a new sitcom called “The Seinfeld Chronicles” aired on NBC. It received good, but not great, ratings for the time, coming in second on the night overall. The network debated whether or not to purchase more episodes; during testing, audiences did not understand this program’s format or connect with its characters, and network executives’ previous experiences told them it would skew to a small, urban audience and never reach national popularity.
On the back of a gut feeling about the show’s potential, one executive took it upon himself to rearrange his own budget, even canceling a planned Bob Hope special, to purchase four more episodes of the show. They didn’t air until nearly a year later, and the show took another three years to become a critical success.
“Seinfeld” went on to air 180 episodes over nine years, and garner the highest advertising rates for any television show, a record it held for five years after it ended. It supported NBC’s programming on Thursday nights for nearly a decade, providing a launching point for programs like “ER,” “Friends” and “Mad About You.” And it took knowing how to interpret, and even ignore, the data at hand and instead understand how the audience and the network’s goals could converge in this new product.
Much has been made of new capabilities — often more imagined than available — to track each and every reader, guiding them to the very thing they most want to see. Companies from analytics firms to ad targeting systems to algorithmic content creators use the “Power of Big Data” to promise hyper-personalized information delivery. And publishers are doing everything they can to optimize their experiences to wring out more clicks and more time-on-site, the two numbers most sought after by their advertisers.
Lost in this discussion, however, are two key problems:
1. This is all about to get a lot more complicated.
Publishers have been struggling with the web ever since the browser was created. Subscriber numbers have fallen, print advertising rates have plummeted, and where publishers have been successful in shifting readers and ads onto digital platforms, the revenue gained usually falls far short of making up the print losses. This has accelerated as mobile browsing becomes more and more important: Right now, fully half of Americans under 30 use their mobile device as their primary Internet device, yet very few providers can find a way to create valuable ad presentations here. This struggle will increase as experiences with smaller, or no, visual elements are increasingly adopted. The industry has barely gotten to a place where it can understand the data behind the current two or three platforms it supports now, and certainly has no idea how to measure a reader’s interest in a report on Google Glass or on the Moto 360.
Further, publication timelines have gone from very discrete – one paper per day, or magazine per week or month – to continuous, where breaking news alerts are pushed before an article is even written; where sites, apps and pages are updated hundreds of times per day, often by algorithms as much as by people; and where readers are in constant motion, changing their needs, context and capabilities in the process. Isolating variables in an environment like this is a superhuman challenge: Did my story get more views because of something I did, or because of some externality? Are more people spending longer on this story because it’s compelling, or because it’s dense or long? If a story gets thousands of views, does that really mean it will have the impact we want it to?
In the far more complex publishing environment coming in the next few years, one metric, or even a handful, won’t be enough. Different metrics by platform, use case, and device – some of which may be more anecdotal or sample-based than census-driven – will be required, and optimizing for those metrics could be as hard as selecting them and measuring them in the first place.
In the far more complex publishing environment coming in the next few years, one metric, or even a handful, won’t be enough, argues Boggie. Screenshot courtesy of Twitter.
2. Even if we get the measurement right, we seem to be missing the point.
In many cases, the organizations struggling most with “engagement” are those that have decades – even centuries – of experience telling compelling stories. Why are they struggling with this new requirement to measure the performance of those stories in simple terms?
Unfortunately, the answer is simple: The critical metric of most journalism organizations is the impact of a story or a service, and measuring that is a very different (and far less scientific) exercise from measuring “engagement” through pages viewed or time spent, and sometimes, measuring impact and engagement are two different goals.
“Time spent” on a story is not necessarily related to its cultural import, its reach, or its ability to change a user’s mind. These are – and have always been – the things we all most want to measure, and the things hardest to measure about journalism. Whether a story is spread through print, websites, mobile applications, or watch-sized glances, it is the quality of the story and the clarity with which it’s told that reaches hearts and changes minds.
It is the quality of the story and the clarity with which it’s told that reaches hearts and changes minds, argues Boggie. Photo by Flickr user Tim Wilson and used here under the Creative Commons License.
The good news, however, is that the remedy is simple, and does make use of new technologies and techniques to reach readers, whether you’re in the newsroom or the ad agency:
Show people the good work you do.
This should be the sole motivation behind engagement, user measurement, audience development, or whatever other buzzword names this practice might acquire. The purpose of learning more about your readers and how they find you is to reach them more repeatably, more often, and to reach more of them tomorrow than you did today.
This should just be called what it is: “marketing.” Publishers want to trade a reader’s time for the information they provide, time that the reader now can spend in dozens of different ways, most of which have nothing to do with journalism. Convincing a reader to spend their time with your piece (article, video, game, podcast, etc.) requires convincing them. Find your readers in other places they read, communicate with them using as many channels as you can in a language they understand, and use the data at your disposal to tailor those efforts.
Do good work.
Marketing to a reader, and convincing them to come read or watch or otherwise consume your work, is a form of a promise: “If you give me a few minutes, I’ll give you something worth your time.” Some of the techniques typically used to “engage” a new audience fracture that promise: Refactoring headlines for better SEO or reposting old content as new because of a timely hook, for example, may draw a reader in with a promise the piece can’t fulfill.
Often lost in the discussion of social measurement tools, audience metrics, SEO techniques and marketing strategies is the importance of the quality of the content itself; this does a huge disservice to what is arguably the single-best audience development technique there is. Making content with which people want to spend time is critically important. Telling stories in new ways, from new perspectives and with new voices, helps build trust in a publisher’s brand so that the next enticement a reader is presented is more likely to be acted upon.
More to the point, however, creating good work is far more a function of the gut than the head. Editors are looked to for their judgment on what story is important to tell, where to give emphasis, how to clearly explain using text, photo, graphics, and video. Data should be used sparingly here, and should be used to refine, not override, the instincts of a newsroom.
The research and investigations we do in The New York Times R&D Lab point to a landscape of increasing reader choice, complexity of presentation, and refinement in data collection and analysis techniques. The organizations that adapt best to these changes will be those that use them to support a clear, concise strategy driven by their mission and values, not those that use them to try and guide their path. In an increasingly complex market, analytics and measurement should be used to refine and correct your message, not to decide what to say.
A version of this article first appeared on the University of Florida’s College of Journalism & Communications’ blog on Medium.
Matt Boggie is an Executive Director in The New York Times Company’s Research and Development group. As the lab is tasked with “looking around corners” and describing the future of media and information distribution, Mr. Boggie is responsible for researching and developing points of view on trends in media production and consumption, identifying phenomena worth further study, and shaping the development of new prototypes and applications the lab uses to educate and inspire.