Government Data: File Not Found

    by Gail Robinson
    August 25, 2008

    Earlier this month the district attorney in Albany, New York, released thousands of pages of documents related to his investigation of a scandal involving former Gov. Eliot Spitzer. This goes back to the days when Spitzer’s alleged improprieties involved misuse of the state police, over-the-top aggressiveness and a lust for smearing political opponents. Not as titillating as the governor’s later transgressions, perhaps, but still of interest to those of us who follow the sad state of government in the Empire State.

    State political reporters salivated — until they heard how DA David Soares was issuing the material. Instead of putting it on-line, the county clerk released a CD and charged $90 for it. “I’m just starting to go through it, and it’s an absolute mess,” complained the LoHud.com blog. “Every page is its own PDF file and not even scrollable or searchable.”

    Soares went somewhat further than most officials but his release that is not a release is all too common. Public officials boast of putting material on-line and then do so in ways that stymie average citizens and even Web journalists trying to review data that should be a matter of public record. Even when material is available, it is, like the Albany documents, often difficult to search and manipulate in a way that provides any kind of meaningful information.


    Those of us trying to find information about New York City government encounter this again and again — despite Mayor Michael Bloomberg’s not so deserved reputation for transparency. In a recent article for Gotham Gazette, Kristofer Ri­os and Joshua Breitbart describe how difficult it can be to get information about city government — despite a path-breaking city law requiring that city documents be posted to the official Web site promptly. “With no method for enforcement, compliance has been spotty,” the authors write. Most information that is available, they say, is in the form of a PDF, “which takes information and turns it into a two-dimensional document that cannot be manipulated or transformed into other kinds of data.”

    And those PDFs can be difficult to find. Last fall, to much fanfare, the city Department of Education graded every school in the city. It boasted all the results would be on-line — and they are. But finding them requires a familiarity with educational jargon, tireless fingers and a second sense about the workings of the minds of Bloomberg’s educrats. Comparing schools, which is what many parents seeking a school for their kid want to do, requires repeated downloads.

    And, as EveryBlock has found, some information is not available at all. The service wants to provide New Yorkers with detailed timely and local information about crime in their neighborhoods. Ri­os and Breitbart report, “In Chicago …, residents wanting to know what crimes have occurred in their area simply type in their address or neighborhood and receive a list of reported offenses. Clicking on the crime brings more information, including a locator map. This comes directly from the police department’s records. By contrast, all the New York EveryBlock site offers is a weekly compendium of crimes by precinct, with no details on the reports and no indication of where in the precinct the alleged offense occurred.”


    This is not for lack of trying on EveryBlock’s part. But rather, Ri­os and Breitbart say, the NYPD’s weekly Comp Stat report does little more than break citywide data into seven broad categories and is only available to the public as a PDF: “The department provides the information by precinct, but city residents cannot search for crime statistics by their ZIP code, neighborhood or even borough.” As a final touch, the link for “Department Statistics” on the police department site has been down for weeks.

    Knowledge is power and politicians or adept ones anyway (and Bloomberg is nothing if not adept) are unlikely to provide unvarnished meaningful information voluntarily. Should civic media, particularly on-line media, play a role in trying to force government’s hand? And, if so, how can that be accomplished?

    Tagged: data government information spitzer

    2 responses to “Government Data: File Not Found”

    1. Sounds like a task for the Online Editors Association
      (which, to my knowledge, does not yet exist – Knight…?)

      in cooperation with the Sunlight Foundation, of course.

      And the most effective way to do it, IMO, is
      a) make it easy (Knight…?)
      b) ask public servants about it when they’re most cooperative, i.e. running for office.
      and then
      c) have the aforementioned O.E.A. do a yearly story rankng our respective communities on this metric, giving awards and booby prizes to the first and last place finishers.

    2. Simon says:

      Public servants have a long history of “making things public” Parrodied only slightly in this quote from the Hitch Hikers Guide to the Galaxy, by Douglas Adams.

      “But Mr Dent, the plans have been available in the local planning office for the last nine month.”

      “Oh yes, well as soon as I heard I went straight round to see them, yesterday afternoon. You hadn’t exactly gone out of your way to call attention to them, had you? I mean, like actually telling anybody or anything.”

      “But the plans were on display …”

      “On display? I eventually had to go down to the cellar to find them.”

      “That’s the display department.”

      “With a flashlight.”

      “Ah, well the lights had probably gone.”

      “So had the stairs.”

      “But look, you found the notice didn’t you?”

      “Yes,” said Arthur, “yes I did. It was on display in the bottom of a locked filing cabinet stuck in a disused lavatory with a sign on the door saying ‘Beware of the Leopard’.”

      There’s more at http://www.planetclaire.org/hitchhikers/

      The kind of unhelpful helpfulness that you discuss in your piece is infuriating. I’d also be interested to know why the SEC allows companies to file on paper. That is really annoying. Especially when you don’t have someone who can go and get hold of the piece of paper and read it to you over the phone.

  • Who We Are

    MediaShift is the premier destination for insight and analysis at the intersection of media and technology. The MediaShift network includes MediaShift, EducationShift, MetricShift and Idea Lab, as well as workshops and weekend hackathons, email newsletters, a weekly podcast and a series of DigitalEd online trainings.

    About MediaShift »
    Contact us »
    Sponsor MediaShift »
    MediaShift Newsletters »

    Follow us on Social Media