Government Data: File Not Found

Earlier this month the district attorney in Albany, New York, released thousands of pages of documents related to his investigation of a scandal involving former Gov. Eliot Spitzer. This goes back to the days when Spitzer’s alleged improprieties involved misuse of the state police, over-the-top aggressiveness and a lust for smearing political opponents. Not as titillating as the governor’s later transgressions, perhaps, but still of interest to those of us who follow the sad state of government in the Empire State.

State political reporters salivated — until they heard how DA David Soares was issuing the material. Instead of putting it on-line, the county clerk released a CD and charged $90 for it. “I’m just starting to go through it, and it’s an absolute mess,” complained the LoHud.com blog. “Every page is its own PDF file and not even scrollable or searchable.”

Soares went somewhat further than most officials but his release that is not a release is all too common. Public officials boast of putting material on-line and then do so in ways that stymie average citizens and even Web journalists trying to review data that should be a matter of public record. Even when material is available, it is, like the Albany documents, often difficult to search and manipulate in a way that provides any kind of meaningful information.

Those of us trying to find information about New York City government encounter this again and again — despite Mayor Michael Bloomberg’s not so deserved reputation for transparency. In a recent article for Gotham Gazette, Kristofer Ri­os and Joshua Breitbart describe how difficult it can be to get information about city government — despite a path-breaking city law requiring that city documents be posted to the official Web site promptly. “With no method for enforcement, compliance has been spotty,” the authors write. Most information that is available, they say, is in the form of a PDF, “which takes information and turns it into a two-dimensional document that cannot be manipulated or transformed into other kinds of data.”

And those PDFs can be difficult to find. Last fall, to much fanfare, the city Department of Education graded every school in the city. It boasted all the results would be on-line — and they are. But finding them requires a familiarity with educational jargon, tireless fingers and a second sense about the workings of the minds of Bloomberg’s educrats. Comparing schools, which is what many parents seeking a school for their kid want to do, requires repeated downloads.

And, as EveryBlock has found, some information is not available at all. The service wants to provide New Yorkers with detailed timely and local information about crime in their neighborhoods. Ri­os and Breitbart report, “In Chicago …, residents wanting to know what crimes have occurred in their area simply type in their address or neighborhood and receive a list of reported offenses. Clicking on the crime brings more information, including a locator map. This comes directly from the police department’s records. By contrast, all the New York EveryBlock site offers is a weekly compendium of crimes by precinct, with no details on the reports and no indication of where in the precinct the alleged offense occurred.”

This is not for lack of trying on EveryBlock’s part. But rather, Ri­os and Breitbart say, the NYPD’s weekly Comp Stat report does little more than break citywide data into seven broad categories and is only available to the public as a PDF: “The department provides the information by precinct, but city residents cannot search for crime statistics by their ZIP code, neighborhood or even borough.” As a final touch, the link for “Department Statistics” on the police department site has been down for weeks.

Knowledge is power and politicians or adept ones anyway (and Bloomberg is nothing if not adept) are unlikely to provide unvarnished meaningful information voluntarily. Should civic media, particularly on-line media, play a role in trying to force government’s hand? And, if so, how can that be accomplished?