Look Beyond Data When Considering New Models for News

My post last month — Future of Local News About More Than Paid Content — generated some thoughtful discussion and comments. But there was one thread that I want to highlight in order to elaborate on an important concept for news innovators.

Before I dive into the details of the conversation, let me summarize my overall point. When it comes to understanding behavior, there are two general strategies. The first is to gather as much data as possible. And in this Google-driven, engineering-led era of product thinking, this tends to be the dominant approach.

The Anecdotal And Observational Approach

But numbers and data can often obscure the important lessons of the way people behave. And that’s why I advocate for the second approach, which is anecdotal and observational. It tends to be overlooked or even dismissed. In the work I’ve done over the past two years, I’ve found this approach to be far more helpful in thinking about the opportunities for reinventing news and information.

My thinking on this topic has grown in part out of a conversation that started on Twitter between myself, Steve Buttry, the C3 Coach at Gazette Communications, and Tim O’Brien, the editor of the New York Times Sunday business section. Steve cited my post to support an argument and Tim replied that my post didn’t prove anything because my analysis was too subjective. He wanted data to support it.

That’s an oversimplification, of course. The series of tweets led to Steve blogging a response here. In the comments, Tim felt his point was misrepresented and explained himself further:

But to extrapolate from Fine’s data to say, as Chris does, and as InfoWeek does, that it shows that newspapers didn’t understand what their readers were paying for is ridiculous. I asked for any empirical data, reader surveys, etc., that outline why readers buy certain papers so we could look at that issue in a less subjective way, not one driven by Chris or InfoWeek’s assumptions. And once we have more of that, then maybe I’ll be proven wrong.

My response is in the comments here. But, again, my message to people designing new services is that there’s another way to think about the problems that need solved.

So, with that in mind, below are some key excerpts from what I wrote. I look forward to continuing this conversation.

Why Data Alone Can’t Solve Our Problems

In trying to think differently about how to deal with the ongoing news business crisis, over the past two years I’ve taken an approach that is intentionally anecdotal and subjective. I simply don’t believe that any amount of data is going to solve this industry’s problems. As I’ve worked on various newsroom reinvention and research projects over the past two years, I’ve come around to believe that the quantitative approach — putting our trust in massive reader surveys, polling data, whatever — has failed us.

Instead, I’m convinced that we need to take a qualitative approach to understanding the behavior, patterns and needs of our communities when it comes to news and information…

Why? Without listing every single study undertaken and tallying all the money spent, I think I can safely assert that over the past two decades, the news industry has spent millions of dollars accumulating data about readers and what they supposedly want. And our industry has responded by altering its products and newsrooms to produce the things that they thought the data told them that readers really wanted. Today, metro newspapers write shorter stories, with faster ledes, and publish more pictures about fluffier stuff. Our leaders have steadily used this data to make decisions that have made newspapers worse every year. Somehow, no one has stopped to consider that no industry has ever solved its problems by making its main product worse. Instead, management points to the data from readers’ survey to insist they’re doing what people say they want. The result is that we’re worse off than ever.

If a data-driven approach was going to solve our problems, wouldn’t it have done so by now? What exactly is the piece of data we’re lacking to begin to address the business crisis the news industry is facing?

I don’t believe there’s a magic data set waiting to be assembled that will lead us to the big “Ah-ha!” I don’t think we’re one reader survey away from figuring it all out. We live in an era where people turn to data as a crutch, leaning on it to give themselves a false sense of certainty. The facts don’t lie, right? Except we know that they do. A lot of such data is formed by the biases and frames through which the questions are formulated, asked, and then interpreted. The newspaper business has failed to recognize its own flawed frames. To this day, no matter what you hear from a newspaper executive, they still believe their primary purpose is to get people to read them in print. It’s why newspapers still spend so much money propping up circulation by subsidizing a large number of people through persistent telemarketing.

My intention, in the original post, was to point out that within the newsroom, these questions have been asked, and continue to be interpreted, through an incorrect frame: The belief that the primary product customers paid for was journalism. It’s not. I do think that in the newsroom, and in the management suites, many in our industry have failed to grasp the need to reinvent the business side. And even among the most experienced new executives, I think there is truly a failure to understand the dynamics of our business and our relationship to the community. While the functions in the newsroom have evolved (not as much as critics say they should, but still….), on the business side, there’s been little attempt to do anything wildly different than what’s been done before.

My perspective on the quantitative versus the qualitative approach to product design began to shift two years ago when I became a member of a task force for a project called “Rethinking The Mercury News.” In the summer of 2007, our executive editor at the San Jose Mercury News charged us with zero-basing the newsroom and re-imagining all of our products and newsroom staffing as if we were just creating the company today. Rather than hunting down piles of research data, or commissioning yet another survey of readers, we decided to conduct the research phase using the “design thinking” process. Design thinking seeks to create empathy with the user of a product by using observation and interviewing to allow you to see the world through their eyes, not your own. The goal is to “re-frame” the issues or problems in the hope of pointing toward different opportunities or solutions.

For me, it’s the anecdotes that provide better insight than the numbers…

The problem with a lot of data we’ve gathered is that you can’t always be sure the people themselves know why they do what they do, or what they really want. Or whether you’re even asking the right questions. During one of my Rethinking interview sessions, my team talked to a woman in her early 40s who spoke at length about how un-interested she was in technology and how she didn’t feel like technology played a role in her life. As she was speaking, she kept taking out her BlackBerry and checking her email. Now, if I’d called her on the phone, and asked her about her interests, I would have checked her off as a woman not interested in technology. But in observing her, I could see that she was. Was she lying to me or was she ignorant? No and no. But she clearly thinks about that topic differently.

To take another example, let’s look at young people and printed newspapers. If there is one piece of data that everyone seems to agree upon, it’s that young people don’t read printed newspapers, right? Its turns out that’s totally false. Over the past two years, as part of the work I’ve been doing for the Knight Foundation (The Next Newsroom Project), I’ve been spending a lot of time visiting college newsrooms, which are far more conservative in their journalism culture and behind the new media curve than professional newsrooms. That was confounding to me for a long time. So what’s going on? The response I heard from college media advisers and college newspapers editors has been fairly consistent: The staffs at college newspapers look around and see all their classmates reading the printed version of the college paper every day. When they get up in the morning, the newspaper bins are empty. If everyone is still reading the print version, why should they worry much about the Internet and all this new media stuff?

As I’ve considered what that means, I’ve tried this experiment a few times myself: Go into the student union and leave a few copies of the newspaper like the New York Times or the Mercury News on a table. They get scooped up pretty quick.

In fact, the generation that doesn’t read print does read a lot of print. What the surveys have really been telling us is that this demographic won’t pay to have the morning paper delivered every day. But when they encounter a printed product that’s free, is compact, and fits the way they consume news and information, and yes, usually has the crossword and comics, then they’ll consume it in large numbers. Do I think print is the future? It’s a part of it, much bigger than most folks believe, I think. How does this square with all those surveys about the news habits of young adults? Those surveys are being commissioned by news executives who are really just trying to figure out how to get young people to pay for the newspaper. They thought they could do this by altering the content. But what they really needed to do was reinvent the product form (compact, free) to fit into these people’s lives (lots of downtime on a pedestrian campus), and that’s a step that’s too radical to be considered by most newsrooms.

These are insights that I’ve gained not through studying the data, but through the subjective, anecdotal approach…In my view, the subjective approach is the strength, not the weakness of my analysis.