Google News to Publishers: Let’s Make Love Not War

Krishna Bharat

In the view of some traditional media execs, Google is a digital vampire or a parasite or tech tapeworm using someone else’s content to profit. As that rhetoric heated up in the past year, Google has responded not with equal amounts of invective but with entreaties to help publishers.

Google launched Fast Flip to help bring old-style page flipping to the web, promoting higher forms of visual journalism and sharing ad revenues with publishers. Then came Living Stories, a new format for updating stories at one URL, designed in tight collaboration with the New York Times and Washington Post. Google realized old-line media were hurting (and lashing out at them), so they wanted to help.

“Specifically for Google News, we don’t see publishers as our competitors. We don’t have a product without their content,” said Josh Cohen, senior business product manager of Google News. “There’s really a symbiotic relationship there. We don’t have a product without high quality content to index, whether it’s on Google News or Google overall. So part of it is there’s interest in making sure that content thrives online. There’s a balance there of the benefit that we certainly get from being able to index the content, and the benefit we give to publishers in the form of traffic.”

I recently went to Mountain View, Calif., to visit Google headquarters, known as the Googleplex, to talk with Google News creator Krishna Bharat, now a distinguished researcher at Google, as well as Josh Cohen, who was in town from New York. Bharat provided background on the origins of Google News (as well as a peek into its future), while Cohen explained how he is spearheading outreach to publishers. The following is an edited transcript of our chat, as well as video clips.

When you first were developing Google News, what did you have in mind? What were your goals?

Krishna Bharat: It was in response to September 11 [terrorist attacks]. I was reading news from a bunch of papers all over the web. And I discovered that there was no efficient way to find coverage of the same topic from different sources. To find the same coverage about the Taliban I would have to go to the L.A. Times site and [go to all these sites]. It seemed fundamentally inefficient. That’s not the way the web was supposed to work. The web was supposed to have a link structure that helped you find content.

Part of the problem is that all of this news was fresh. By definition, news is fresh and doesn’t have links. And if Google is to fulfill its mission to find information efficiently, it occured to me that what I was doing a computer could do. A computer could, in fact, visit all these websites, find the same article, or similar articles, and group them together. I tried it, and it worked.

Also, given my background, having grown up in India and read about Western events from there, I knew the diversity of reporting that existed, and certainly different points of view. Especially on this subject [around 9/11], there is a Middle East point of view, a British point of view, an American point of view. Bringing those views together seemed like a good social function. Helping people understand multiple points of view, and hence becoming wiser for it — whether they agree with it or not — just understanding there is another point of view is enlightening.

Bharat describes how Google tries to serve the user first and then figure out the business model later:

How do you measure the success of Google News?

Bharat: We look at the number of queries that we impact on web search. I don’t remember the number now, but it’s a non-trivial subset. It’s also a sign of the times, there’s a lot of interesting stuff going on, a lot of good real-time information. The fact that we are contributing to that, making web search more powerful, and we’re satisfying user needs, it’s a sign of success. Besides that, the headline pages we have are a starting point for some people, and they follow the links, and we send traffic to publishers, which is also very satisfying.

I remember when Google News first launched, you made a point about saying that there was “no human intervention” in creating the site. But humans created the algorithms and have had a lot of intervention in it, right?

Bharat: Of course the algorithms don’t come out of the blue, but that’s obvious. The interesting thing is that the algorithms aren’t trying to replace an editor, they’re trying to assimilate the wisdom of mulitple editors, and say ‘statistically, this is the most interesting story right now because more editors have covered this story than any other story.’ That’s the basis for our ranking. Even in news search, we look at who’s publishing a story, when it’s published, but also how big of a story is it. Ultimately, we are aggregating editorial wisdom.

I wouldn’t say at all that this could operate without humans. In fact, everything on the web is a function of human output. People author content, people link to content, people prioritize content. And all of the different algorithms on the web, be it web search or Digg or something else, is drawn from human input. So yes, we draw on human input but the algorithm ultimately decides what leads and what does not lead.

Josh, what about in your role working with publishers? How do you measure success?

Josh Cohen: If we don’t have a successful product, then it’s not going to be all that successful for publishers. The business model we have is a little different. We’re an aggregator, but we’re not really a portal. So our focus is to get all that traffic and send it out to publishers. The more that we grow, that means the bigger our traffic hose is from the links that we’re sending directly to publishers. So if we don’t have something our users want, everything else falls apart.

On the engagement side, we don’t have any content to offer pubishers — we don’t have editors or reporters — but we have technology and tools. We see publishers taking advantage of the tools we have to make their websites better. Probably the best example today is Google Maps. So many editors will use the open API, embed that into their stories, think of different ways of telling stories online that you can’t do in a paper. And the last part is monetization, which is a big part of Google’s business, whether it’s in display ads or search ads to help them make more money.

Cohen talks about the legal issues surrounding Google News, and how Google lets publishers remove their content from indexing:

How has Fast Flip gone so far? I know there’s a revenue split with publishers, are they happy with that?

Bharat: Fast Flip was a way to increase engagement and look at new ways to monetize that. When I pick up a magazine, turning the pages is instantaneous. It’s really fast to turn pages and see a lot of pages rapidly. On the web, things are slow, but they shouldn’t be slow because we have the technology to make it fast. Loading a page from a top news sites may take 5 to 8 seconds on broadband. If it took you that long to turn the page of a magazine, you wouldn’t turn many pages.

There’s inertia here, so we’re decreasing inertia and allowing people to see more content. We made Fast Flip really fast so you can skim through content really rapidly, and in the process encounter a lot more ads, thus making more money for the publisher. When you find something interesting, you spend time on it and click through. So we have a site optimized for skimming, but even the skimming experience is monetizing for the publisher.

And are those ads sold as CPM (cost per thousand) ads or CPC (cost-per-click)?

Bharat: They’re CPC ads, but we’re just starting out with this experiment. Right now we’re serving the ads, but you could see a situation where a publisher serves the ads. Or you could see a situation where this is premium [pay] content and the idea is to encourage people to buy the content and buy subscriptions. There are any number of ways that this could evolve. The idea was to find out more about user behavior if you made it really fast. And we learned that people look at a lot more content, and a lot more ads.

We also found out that the old model of just showing you a title and a snippet [on Google News] does not do justice to certain kinds of content — very visual content or enterprise journalism that if you don’t have a sense where it’s coming from and that the Economist or the Atlantic are behind it, you don’t appreciate the quality of the content. The title does not do it justice. We’re observing that a lot of traffic is going to sites that are extremely well typeset and carefully authored. And right now the model on the web does not help that kind of production.

Cohen: The assumption we had going in was that if you’re showing more content, then there would be a lower clickthrough rate than showing just a headline and a snippet. The assumption was that with a lower clickthrough rate people would consume more of that content. It’s hard to find that kind of content because Google News is so search-based. This is more a factor of serendipity. You don’t necessarily know you’re looking for these long-form investigative pieces that [you experience] more like sitting back and reading a magazine.

And we’re seeing now that, especially for smaller publishers, Fast Flip is giving them a real burst of traffic. And Google News is so focused on breaking news that this is really a new channel for them.

Bharat and Cohen discuss the Living Stories project, and how Google employees were “embedded” in the New York Times and Washington Post newsrooms:

Where do you stand on what can be included as a source that’s indexed on Google News? I remember discussions about whether blogs should be included, but there are also press releases, too.

Cohen: Overall, I’d say the bias is toward inclusion. Increasingly, it’s a gray area, but in the same way Krishna talks about having the algorithms drive our rankings, we don’t want to sit in judgment saying ‘this is a good source or a bad source.’ Making qualitative judgments is not a place where we want to be in selecting the sources. So you really try to make it a binary decision. Is it current events? Is it covering news? That’s a big one. Is there original content? If you’re aggregating content you want to get it from original sources instead of from sites that are purely aggregators.

We have press releases and label them as such. We have blogs and label them as such. A big part of it is having a certain level of disclosure for the user so they can understand the nature of the sources.

At one point, I remember that you would include blogs but only if there was more than one person working on it. Is that still the case?

Cohen: We want some evidence of an editorial review process. But it’s not easy and it’s getting that much more difficult to define those kinds of sources. There’s a larger debate about what is news and not, and whether Twitter is news or not. I have a feeling it will only get more difficult.

Bharat and Cohen talk about possibly integrating real-time feeds from Twitter into Google News search, and the challenges of doing that:

Have you been tempted to bring in editors and even fact-check what goes onto Google News?

Bharat: Just the sheer volume of what we deal with becomes challenging, and then there’s the issue of objectivity. If we had an editor in-house, then we would become another publication. That said, there are editorial functions one can perform that stop short of making those decisions, that don’t take away the diversity we have right now. It is something we could think about in a limited scope at some point in the future, but right now we don’t have editors in that role.

Can you talk about some projects you’re working on now, anything coming up with publishers?

Bharat: What I can say is that the industry appears to be moving toward pay walls and subscriptions. And we’ve explained that Google as a company is very interested in working with whatever scheme ultimately takes off. We are happy to bring technology to bear on the problem. If a publisher feels they can monetize their content with ads, more power to them, we’re absolutely happy to work with them, helping them drive traffic and providing increased engagement and better monetization models for ads.

If they do want to put the content behind a pay wall, you still need to find the content in order to get subscribers, and we’re happy to play a role in that. Google would still want to link to the content or a preview of it and still drive traffic, which means we’d still have to know where the content lives, and there are technical challenges there. Beyond that, we have ways to pay for content like Google Checkout. We are actively looking at ways we can work with the industry to help non-ad based solutions take off.

The other thing we have a broad interest in is personalization. Every time a reader looks at something and says ‘that’s not for me’ and moves on, there’s inefficiency in the system. Along with getting the top news of the day, we want to make sure the rest of their experience is as efficient as possible — not only on Google News but on other publishers’ sites. Trying to be smart about selecting content will help the industry, and that’s something we’re investing resources to try to figure out how that can be done differently. And when we have technology that’s ready, we’d be happy to work with the industry to make that successful.

Would you go by what users input or by their browsing history?

Bharat: A bit of both. Obviously if users are willing to tell us, that’s great. It’s very accurate. Beyond that, there’s plenty of evidence from the way they browse the content to tell us where their attention is going.

Cohen talks about some of the factors leading publishers to attack Google, and how they deal with that heated rhetoric:

*****

What do you think about Google’s efforts to work with publishers? Do you think publishers should work with Google to help with their businesses online or go it alone? Share your thoughts in the comments below.

Videography for this story was captured by Charlotte Buchen.

Mark Glaser is executive editor of MediaShift and Idea Lab. He also writes the bi-weekly OPA Intelligence Report email newsletter for the Online Publishers Association. He lives in San Francisco with his son Julian. You can follow him on Twitter @mediatwit.