URL Shorteners Help Track Links, Take Heat for Framing

If there’s any doubt that an online titan can be easily overthrown, look no further than the URL shortener Tinyurl.com. For years, it was the most popular of its kind and the dominant (and default) URL shortener for Twitter. Then a few months ago I began to notice that it had all but disappeared from my own Twitter feed.

With the prevalence of micro-blogging and character limits on posts, the URL shortener field has quickly expanded into a vibrant marketplace, with big players and individual sites creating their own shorteners. The industry has even faced controversy recently when Digg’s Diggbar, what many had considered a URL shortener, began redirecting users to Digg’s front page. Though this wasn’t the first time the issue was brought up, it left many questioning the permanence of such links. Would a shortened link you created still exist in its original form a year after you posted it?

Lasting links

Andrew Cohen, general manager of bit.ly, stressed that users should be aware of all these things when they’re choosing which shortener to use.

“A lot of the players in this space have been one or two person companies, very small players,” he told me. “What we wanted to do was set a very high standard for reliability. So it was really important to us that our short URLs always redirect, and so we engineered multiple layers of redundancy…so even if there was a disaster, the bit.ly short URLs would still redirect.”

Cohen explained that shortened URLs often mask the real web address, so there’s a need for underlying transparency. Because of bit.ly’s API, certain Twitter applications allow users to see the real URL when they hover over the link, and the service has also incorporated spam protection to try to warn users when they’ve clicked on a suspect link.

“Another important thing is permanence,” he said. “So we never recycle a URL. I believe that a couple other small URL shorteners reuse the hashes they generate. We don’t believe in that for a couple reasons. We feel the full credit for the link goes to the publisher of the underlying content, so we can only use one redirect. We also don’t want to allow any kind of bait and switch scheme, for instance someone who has fabulous content on the first click and then switches out to malware so when you return it’ll take you to that.”

Of course, what bit.ly is most well known for is its real-time metrics, something that Cohen said is much more difficult to maintain than the URL shortener itself. Not only are individual users able to track how many people click on their links, but the company counts the clicks on all its links in a given week. Back in May, it was averaging around 100 million clicks a week; now it’s up to 230 million.

Framing

One of the things bit.ly purposely avoided was so-called framing — when a URL takes you to a page that contains a bar at the top from a website other than the one containing the content. The DiggBar caught a lot of heat for this, with critics saying that it punished the original content creators by masking their URLs. But Ryan Holmes, the CEO of Invoke, the company that owns the URL shortener ow.ly, said that he decided to use framing because it enhances the user experience.

Ryan Holmes

“We kind of looked at what the people’s issues were with it, and we addressed all those issues,” Holmes said. “The biggest complaint was around search engine optimization. So we addressed a lot of the issues. We give full attribution to the content owners so Google will reward the original content source. It’s got a benefit for the person who creates the URL, because their avatar is on the page. In the social bar people can see who originally created it. For the actual content owner there’s a great benefit of having the bar on the page, because we see a doubling of URL clicks. We’ve tried testing with and without the bar, and with the bar you’ll see maybe twice as many clicks because of the built-in retweet functionality.”

He explained that ow.ly grew as a product to complement Hootsuite, a service described by its website as “the ultimate Twitter toolbox.” I asked him how the tool’s analytics differ from bit.ly’s.

“Bit.ly has some metrics but you have to be careful of bit.ly because all your analytics are exposed to the public, and not a lot of people know about that,” he said. “So the risk there is if I’m HP and Dell is using a ton of bit.ly URLs, I can go and look at the Dell shortened urls on bit.ly and see what’s getting the best clickthrough, what time of day, what specials resonate well, and learn a ton about my competitor, and that’s a big risk of bit.ly.”

Custom shorteners

Though most people use these more general URL shorteners, a few sites have created their own custom shorteners to promote their content. If you click on the Twitter link on a Gawker Media blog post, for instance, you’ll get a pre-written tweet with the headline and a shortened URL using Gawker’s domain. I spoke to Thomas Plunkett, who heads Gawker’s engineering, and was surprised that the function wasn’t even originally created for micro-blogging, but for email.

“The problem we were facing several years ago is that people were trying to send our links via email, and one of the common problems that URL shorteners try to address is that the URLs would sometimes break, get cut in half,” he said. “So what we introduced was switching them around. So effectively we’ve had the URL, gawker.com-slash-whatever you want to bring up. It has been working for three years now since we first did it.”

Gawker didn’t introduce the URLs into its tweets until about two months ago, and since doing so Plunkett said he hasn’t noticed any direct increase in traffic because of it. But he said there are likely plenty of indirect benefits. For one, it improves Gawker’s branding, because its URL is included in the links. It also gives users at least some idea of what they’re about to click on, whereas most shortened URLs give no indication.

When Digg introduced its DiggBar, it reportedly saw as much as a 20% increase in traffic to its own site, indicating that there are benefits to creating your own short URLs, especially ones that use framing.

I reached out to Digg for an interview and only got this statement:

The goal of DiggBar and Digg short URLs is to enhance and streamline the Digg experience for our users. If folks are sharing stories via Digg and the DiggBar, we want users to have the ability to view comments and related source content on Digg.

I didn’t get a chance to ask about the criticisms Digg has faced about how it redirects its traffic, but the company has made several changes since launching the tool to appease critics.

Monetizing Shorteners

But how do these URL shorteners plan to monetize these tools? For bit.ly, Cohen said that the key to monetization is the click metrics. With all that data, he said, he is able to see news stories gain traction well before the mainstream media begins to cover them. He didn’t go into too much detail, but he outlined to me an ad-supported site that would rank stories by rising attention, a kind of Digg-like aggregator.

“We’re seeing more than a billion clicks in the course of a month,” Cohen told Wired recently. “Looking at that volume of data, we can see the most interesting and the most important content that is being shared across the whole of the real-time web. Sometimes that’s humorous stuff — the other day, the most shared video we saw on the web was William Shatner performing a dramatic reading of Sarah Palin’s farewell address.”

As for ow.ly, Holmes described it as simply a tool that would help in selling the larger HootSuite package, making the URL shortener a kind of loss leader to bring in more customers.

Of course, sites like Gawker and Digg are already monetizing their sites. For them, creating their own URL shorteners are simply a means to get a bigger piece of the micro-blog traffic pie. As Twitter receives millions of new users every month, it’s a tool that web publishers can’t ignore.

Simon Owens is a social media consultant and associate editor for MediaShift. For more about him read his blog or contact him at [email protected]