What It Is
The New York Times’ Represent is a data aggregator and sorter that points to information about elected representatives in New York City. If New Yorkers enter an address, they can see their political districts (Congressional, Assembly, Senate and City Council) and representatives. Represent will also track what their representatives have been doing through a recent activity feed from NYT articles and congressional votes.
In a city like New York, with over 8 million people and more than 150 elected representatives, keeping track of the people that speak for you can be difficult. Represent is a solution to that problem.
Why It’s Innovative
Represent is a look at the future of online journalism — focused, local and geographically relevant. It’s a different way to group and browse information based on an individual’s political districts.
Some have compared Represent to EveryBlock. It does fill a hole in EveryBlock’s coverage, taking the concept of block-by-block news and expanding it to fit the political realm of information. In fact, EveryBlock recently hooked up with The New York Times to display political news items for each block.
Who’s Behind It
Represent was put together by Andrei Scheinkman, Derek Willis and Stephan Weitber.
Scheinkman is a software engineer in the New York Times’ Interactive Newsroom Technologies group. He came up with the idea for Represent in response to an internal development contest. The winning project would get the resources for further development. The project received honorable mention, but the group adopted it as an official project anyway.
Scheinkman said his ideas for Represent came from a project based in Chicago called CivicFootprint, a site that uses street addresses to help people find out who their representatives are. He worked at CivicFootprint along with Paul Smith, who now works at EveryBlock. Scheinkman was also inspired by Facebook’s activity feed and Twitter.
To this concept, Scheinkman added maps of the various districts to which an address belongs and information about the representatives and their activities.
Scheinkman’s programming contribution to the project was the logic that figures out what to show a user. After aggregating Times articles, blog posts and voting data, the elements then had to be filtered to show a user the information relevant to their political districts.
“In particular, I was interested in applying the idea of ambient awareness to our relationships with our elected officials,” he said. “Last September, I read Clive Thompson’s article about the concept in the Times magazine and was struck by this passage: ‘Each little update — each individual bit of social information — is insignificant on its own, even supremely mundane. But taken together, over time, the little snippets coalesce into a surprisingly sophisticated portrait of your friends’ and family members’ lives, like thousands of dots making a pointillist painting.’
“It occurred to me that by collecting little updates about the people who represent each of our readers — a mention of every time they miss a vote or attend a committee or say something that appears in a blog or an article — we might be able to create a similarly sophisticated portrait of their elected officials. Even though each item that appears in Represent might not be that interesting by itself, in the aggregate they provide the reader with a pretty detailed picture of what their representatives do.”
Scheinkman also set up a geocoding service that takes the address and converts it into a point (latitude and longitude). That information is used to place the address with the various political districts.
Basically, a database query finds out which districts contain that address, and then returns the representatives for those districts and Times stories and blog posts about those people.
Derek Willis is another member of the Interactive Newsroom Technologies group, which is a collection of journalists/developers in the newsroom working on all kinds of tech projects.
Willis works from home most of the time, and is in New York about once a month. He was intrigued by the idea of working with GIS data, which he hadn’t done much before. His job was to gather the map files for New York City, set up GeoDjango and create the basic outline of the application. Then he compiled information on the individuals representing different parts of New York in the City Council, State Legislature and Congress.
Stephan Weitberg, a front-end developer, and Elliott Malkin, an information architect, also contributed to the project.
Challenges
Scheinkman told me that even though Represent didn’t win the internal development contest, Aron Pilhofer, editor of interactive news, is open to new projects and really liked the prototype.
There were some other obstacles, however: Because of the election, Scheinkman and Willis were too busy to work on Represent for a few months. Then there was trouble with getting voting data; neither the State Legislature nor City Council publish voting information in an easy electronic format.
Willis had not worked with GIS data very much, and Scheinkman hadn’t worked with Django, so there were some technological challenges to overcome as well.
“We tried to keep Represent fairly simple so we could finish it in time for the contest and so it would have a fairly immediate and obvious appeal,” said Willis.
3 Questions
Most New York Times projects are built with Ruby on Rails. What was it like to work with Django instead?
Willis: Some folks have asked how we ‘managed’ to use Django at the Times, which uses Rails for most of these types of projects. And it was simply a matter of personal preference — the contest allowed entries built with pretty much any technologies — and there’s a keen interest at the Times in using the best tools for the situation, and being flexible.
They’re both excellent tools that have strengths and weaknesses. I don’t see a problem in using both; in fact, knowing one has helped my understanding of the other. I’m more interested in people learning one or the other than in telling them which one to learn.
Can you tell me more about the internal development contest?
Willis: Originally (in 2007), it was open to employees who worked on the digital side of the company, but last year they expanded it to include any employee. People proposed various concepts that ranged from mobile applications to improvements in internal workflow. It’s a really good idea; I wish more media companies did it.
The winners (there’s usually more than one) get a prize and a commitment from the company to see their projects come to fruition. The judges include people like Marc Frons, the CTO of digital operations, and other luminaries in the company.
What kind of response have you got since launching the beta?
Willis: We got some great feedback from users — requests for additions and suggestions for improvements. Other people have wanted to know how we did it and whether it would be expanded to other areas.
Scheinkman: I don’t look at traffic numbers because I don’t want to judge projects by the amount of traffic.
Specs
Being a Django app, Represent is written in Python, but the geocoder was built using Perl. The app uses Postgres as the backend database and the PostGIS spatial extensions to do the heavy-duty GIS work.
The votes come from the Times’ own database of Congressional activity and the site is deployed on Amazon’s EC2.
Check out more details on the technology behind Represent on the New York Times’ Open blog.
look into the estell richardson murder case bush is involved check gus puyer and get the whole scoop i have proof bush is a racist this is not a hoax and look into his ties with torture
The New York Times ‘Represent’ is a terrific tool! The GIS company I work for has been receiving requests for a tool like this, but for local info **nation-wide**. We’ve therefore built an address-based legislative district matching and elected official web API, called Cicero: all levels of legislature + district boundaries — including more than 100 local gvts. We also have data for Canada, Australia, and New Zealand, working on France now and many more foreign legislative systems. http://www.avencia.com/Cicero/LiveSample.aspx
We certainly hope that more geographic tools like these will help open government efforts as well the newspaper industry that is in so much need for a new direction.