How many different kinds of errors is it possible for journalists to make? And how would you classify them or organize them into useful categories?
These questions are not my attempt to concoct a tactful paraphrase for “How many different ways is it possible to screw journalism up?” Rather, they represent one of the interesting issues we face as we move work on MediaBugs from the project-organizing phase to the “let’s build something” stage.
There’s a wealth of established practice in the software field for the kinds of data you can associate with a bug that a user finds in a program: how important the bug is, where the bug is located, how work on it fits in to the rest of the project, and so on. In software development, the purpose of the bug tracking system is, mostly, to define and organize the work of fixing bugs.
As we attempt to apply this model to the world of journalism, we find little in the way of similar established practices in our field. Individual news organizations sometimes track their own errors internally, but, as far as we’ve been able to determine, there is no common, industry-wide nomenclature for categorizing those errors — no Library of Congress classification or Dublin Core metadata standard.
We’re pretty much on our own. So we’re doing our best to devise an initial set of categories, knowing that we’ll probably need to revise them once we get real data from real users. (We’ve already drawn much from the invaluable work of my colleague Craig Silverman, in his book Regret the Error.)
Here’s the list of categories we’re playing with right now:
- misquotation
- mistaken identity
- other simple factual error
- ethical issue
- faulty statistics or math
- error of omission
- typo, spelling, grammar
- other
I’d love to hear what you think of this. Have we left out something obvious? Is this valuable or interesting?
Any set of categories will need to meet two goals:
- It should make sense to users who are trying to make quick decisions about categorizing the errors they’re reporting.
- The breakdown of the total universe of errors that the list provides should ultimately be useful as we try to understand why errors happen, and how we can minimize them.
We know that there’s no bright, shining line one can draw between errors of objective fact and subjective problems with media coverage. Errors don’t fall into two distinct buckets labeled “fact” and “opinion”; there’s a spectrum between the two.
We want MediaBugs to favor the “fact” side of that spectrum, so our choice of categories is weighted in that direction. I believe this is where we’ll find the most common ground between journalists and the public, and make the fastest progress in our effort to bring the two together. We’ll know a lot more soon!
View Comments (17)
I am not sure how this would be classified but simply taking something someone says as fact. For instance, someone might say "so and so study shows this so cigarettes really are good for you." The quote and the attribution would be correct but, if one were to check, one might find that the study really didn't show what the person says or there are so many biases in the study (who paid for it, etc) that the study is pretty much worthless. Yet, such statements are frequently given equal time to the other side in the interest of "fairness".
I am currently an out of work journalist, and I agree that in the past I have a made a few mistakes. I do my best to make sure that I do not. I double and triple check stats/figures, if I use them in a story. I want to make sure I perform my due diligence. I currently live in a small town with a small town newspaper. There are quite often mistakes in their reporting because they don't have a fact checker or copy editor. The staff writes, edits, and lays out the paper, and thus mistakes are often made. I have applied to work there, and I hope to improve it with my writing style and attention to detail that they sometimes fail in.
This one from Muntain is very helpful to me as a writer so thanks for adding that.
For me, when you turn your stuff over to the editor who then changes something either intentitionally or unintentionally and makes you look like a dumb a**, even something minor like putting the quote marks in the wrong places or changing your title.
We've been talking about this at Planet 3.0, with respect to debugging the press's spread of climate disinformation.
Scott, IMO you might want to collect a bestiary of press misfires before settling down to categorize them; so let me contribute one that we can think about.
This one had cascading flaws in the trajectory of its development and spread.
(And what we see as "bugs" are seen as "features", by interests that *don't* want the press to shoot straight - so they're ripe for exploitation.)
Here's how it works:
Step 1: Someone with a credential but without relevant expertise writes a schlocky study with nefarious-entity-friendly conclusions.
Step 2 works right: the study gets panned by experts. ("worst I've ever seen", etc)
Step 3: a local-paper story about the study and its reception errs with false balance, and doesn't get to the experts' criticisms until Page 2.
(But it does mention them in headline, and provide a gentle reference in the lede.)
Outcome: 50% of readers are misled.
Step 4: the national bureau picks up the story, and - playing Telephone - replaces the "Some criticize study" headline with a credulous "Credentialed Person raises alarm" one, and only displays (the pre-criticisms) Page 1, linking to the local paper for the rest.
Outcome: rarely will readers bother to click through; 95% are misled.
And finally, the coup de gras -
Step 5: the local paper stashes all but Page 1 (the "happy" page) behind a paywall.
Outcome: rarely will readers bother to pay up; 99% are misled.
So, 99% of readers who stumble on the story get a take-home message that's 180 degrees from reality, and it's nobody's fault.
Scott, I know you're familiar with some of Jay Rosen's criticisms of media. I think one of the main issues that he brings up, if I might paraphrase, is unrecognized or unacknowledged bias. Simply by subtle choice of words, the selection of interviewees, the "framing" of a piece, the image conveyed to the mind of the reader may be strongly influenced away from what others might see as objective reality, toward the "reality" the journalist sees.
Some of this is very blatant - Nedra Pickler's reporting on Democratic presidential candidates for example. A lot of it is far more subtle. A good picture of it, which again I have to attribute to Jay Rosen, is the concept of the "sphere of legitimate debate", vs. spheres of consensus and an outer "sphere of deviance":
http://www.huffingtonpost.com/jay-rosen/audience-atomization-over_b_157807.html
That is, the most important source of journalistic error is bad "sphere placement": writing articles that imply that totally discredited ideas deserve "legitimate debate", or conversely treating perfectly legitimate debatable subjects as completely off-limits, in the sphere of "deviance".
How do you correct those bugs? I don't have a clue where to begin.
How do you define what constitutes a journalistic error? How do you avoid the trap of focusing on easily-measured yet insignificant errors, compared to more major ones that setiously disinform the readers?
One type of error would be a large-scale error of coverage omission - e.g. the journalist deliberately avoiding covering a whole class of stories on his beat that would have given readers the information they need to be free and self-governing - instead allowing them to remain misinformed, and/or become misinformed by others.
Or, worse yet, actively misinforming by serving as an "agent of influence", playing in the disinformation symphony by reporting little, true things that nonetheless push the public toward misunderstanding the big picture.
Another major error: printing disinformation in the non-news sections of the newspaper (letters to editor, op-eds, advertising), without making it clear to readers that what they read there is not to be believed (after all, if you did successfully convey the don't-believe-it msg to readers, there'd be no incentive to buy the ads or send in the letters&op-eds.)
Errors of obscurantism(?)
e.g. an "ask the newsroom" Q&A where the Qs to be answered are selected by the askee, and unanswered Qs are (quietly) never shown.
or displaying user comments in an unfiltered, or ineffectually filtered, form; ensuring that the gems stay buried amongst the dross.
Moderating comments by enabling the Pluck feature that quietly makes one's comments visible to oneself but not to anyone else.
Thanks again for all these thoughtful comments -- plenty for us all to chew on here!
Our mandate and mission at MediaBugs is explicitly to try to concentrate on *correctable* problems or "bugs." What that means in practice is something we're going to hash out as we go along. But it does, I think, impel us away from issues of bias and subtler kinds of informational malpractice -- not because those aren't worth exploring and exposing, but because there are already institutions and fora devoted to that dialogue.
For example, with Anna's example of the discredited study: there's a role for MediaBugs to play in this scenario, and it is simply to air the discussion about that study. Was it accurately represented in the story or not? And if not, should the publication correct its coverage?
We recognize that there are going to be plenty of people who will be disappointed by this focus. Our hope is that (a) there already, thankfully, exist on the Web a vigorous and broad-based culture of general media criticism; and (b) by creating a forum for working out specific, correctable issues, MediaBugs can start reversing the climate of distrust between the media and the public.
We'll learn a lot more, of course, once we have a service ready for your use!
jfnbjdasfbujsd
Regarding the category of faulty statistics and math, if you haven't seen them, you should look up Mark Liberman's postings on Language Log about how to present statistics. You can start with this post, http://itre.cis.upenn.edu/~myl/languagelog/archives/004992.html, and then wander around