If a technique were proven to reduce errors, don’t you think journalists, journalism organizations and journalism educators would race to embrace it? To demand that journalists follow it? To teach it in J-schools?
Well, as Craig Silverman has pointed out repeatedly, the checklist is the most effective system of preventing errors, so effective that pilots and surgeons use checklists routinely when they fly and operate. (When I had a recent biopsy, I noticed the surgeon and nurses clearly following checklists as they confirmed the site for the procedure before anesthetizing me, checked my identification bracelet and asked my name and date of birth.)
I was unsuccessful in persuading the Society of Professional Journalists to call for use of checklists in the 2014 update of its Code of Ethics. But I think journalism professors should advocate checklists in teaching basic reporting and journalism ethics. Why should journalists not adopt the best error-prevention system?
I first heard Silverman make the case for checklists in an accuracy workshop he led in 2010. He noted that veteran pilots use checklists. US Airways pilot Chesley Sullenberger, who became so famous we knew him just as “Sully,” used a checklist for his 2009 emergency landing in the Hudson River, Silverman noted. When the discussion shifted from pilots and surgeons to journalists, it started getting uncomfortably familiar. One of the most common sources of error, Silverman said, is the veteran journalist relying on experience.
About the time Silverman had me won over to his checklist point of view, he said something surprising: He has frequently encouraged journalists in workshops and on his blogs to develop their own checklists, but no one had ever told him that they’d done it.
So I developed my own checklist, which includes several items from Silverman’s checklist, but I changed a few items and added others, as he had encouraged. Here’s my checklist, explained in more detail on my personal blog, The Buttry Diary, where I initially published it in 2011:
While Reporting
- Ask sources to spell name and title; then verify what you wrote
- Record or transcribe interviews
- When someone cites numbers, ask for (and check) source
- Ask “how do you know that?”
- Seek documentation
- Verify claims with reliable sources
- Save links and other research
- Ask sources what other reports got wrong
While writing
- Note facts that need further verification
- Cut and paste (with attribution) quotes from digital documents.
Photo by Nic McPhee and reused here with Creative Commons license.
Final Checks Before Submission
- Numbers & math (have someone check your math)
- Names (check vs. notes and one other source)
- Titles (people, books etc.)
- Locations
- Compare quotes to notes/recording/transcript
- Check attribution (insert link if from the web)
- Definitions
- Verify URLs (check them and check whether cited content is still there)
- Phone numbers (call them)
- Spelling & Grammar
- Spellchecker Errors
- Have you assumed anything? (If so, verify, hedge or remove.)
- If you have any doubts, recheck with the original source.
- Where your understanding is weak, read the final copy to someone who does understand.
When finished
Correct any errors you found in your archives, databases or other resources you control (but be certain you have verified the new information).
Checklists are worth the time
The most common response when I posted this checklist in 2011 and as I’ve advocated checklists in discussions of journalism ethics standards is that no one has time these days to use a checklist. I don’t buy it. Accuracy is more important in journalism than ever. Wherever we go in developing new business models for news, trust is always central to the value we want to provide. It always has been, and deadline pressure is nothing new to journalism.
Don’t be daunted if my checklist (or Silverman’s or yours or a student’s) looks time-consuming. Some of these checks will take seconds. For most stories, running through a checklist takes only 10 to 15 minutes, if that. You simply work the checks into the workflow of your reporting, writing and fact-checking. The doctors and nurses checking my date of birth took only seconds, but ensured they were operating on the correct patient.
Photo by Nic McPhee and reused here with Creative Commons license.
Ensuring accuracy is already part of a journalist’s workflow, and many of the steps outlined are already followed on most stories; the checklist just makes it more consistent and rigorous.
If you’re under deadline pressure, you can check the first few paragraphs and post that, saying the story will be updated. Then check the rest and update. Accuracy is worth a few minutes, because the damage an error causes lasts way longer than the delay a checklist causes.
Two newspapers published errors in stories about me in 1991. I remember those mistakes more than two decades later, and they were preventable. I don’t want people remembering my errors that long.
So what’s your checklist? What will you add to Silverman’s and my lists? Would development of a checklist be a good assignment for your students? Silverman recommends printing out your checklist and laminating it, so you can use it again and again. If it’s good enough for Sully, it’s good enough for me.
Steve Buttry is Lamar Family Visiting Scholar at Louisiana State University. He has spent more than 40 years in the news business. He blogs at The Buttry Diary. On Twitter he’s @stevebuttry.
Steve Buttry’s post led to a great conversation on social networks, which we have appended below.
View Comments (9)
Editors, not just the reporters, need to keep this check list in mind. Thanks for the post!
The most important standard question is "how do you know that?" in my view. Editors need to ask their reporters this question all the time as transparent sourcing is going the way of the buffalo with news turning into more-like commentary even at the biggest news outlets. For example, our reporting at iMediaEthics found it is not usual at the NY Times to have zero sources for a news story about Egypt!
Your other point caught my attention: "Ask sources what other reports got wrong." This is a standard question for iMediaEthics as as errors are part of our beat. However, I would think it was a non-standard question for normal reporting. What made to add this and what is a reporter to do with the answers?
In academics other peoples errors are part of contextualizing one's on work, not so in journalism --although I think it is a great idea to do so.
It has always been a worry for me. If one news outlet reports 10 fire trucks on a fire scene and another reports there were only 5--how does the public know what is accurate or trust the media for getting it right? One or both counts of trucks could be wrong without the transparent context of an assertion that your news outlet got it right and here is why [using statement from Fire Dept, photo of scene etc.].
Thanks for your comment, Rhonda! Of course, both reports about the firetrucks could be correct, just at different times. Or one might be counting actual firetrucks and the other total emergency vehicles. In addition to being accurate in details, we want to be precise in our language and accurate in context.
Case in point, the reporter may do a count at one moment but the official count comes from the the fire dept. They will provide the correct number of total number of vehicles that responded to the particular fire. This can be confirmed by dispatch/ 911 computer records. Hence why no valid excuses to get this wrong and it is a fact check fail. (Rule of thumb in NYC FDNY: Each "alarm" adds 4 engines and 2 trucks--so the recent 7 Alarm Brooklyn fire would be approx 35 engines and 14 trucks dispatched, plus other special vehicles as it is such a big fire).
So Steve, what should reporters do with the information that sources provide to them about other media outlets' errors?
Should they specifically cite the errors they learned
from a source in their own reports?
I was once told by a NY Times reporter that all he cared about was that he got it right when I asked him why he didn't tell the public his information was correct --not the competitor's. But how does the public know he is right and not all the other reports unless he says why and how his information is correct and theirs is wrong ?
Do you see this reporting step in the check list ("Ask sources what other reports got wrong") as part of standard reporting or something you (and/or Craig) suggest? I love it either way as it adds both accountability and context for the public.
Official reports or estimates such as you describe can have errors. For instance, some of the 35 engines and 14 trucks in your example might have been out for maintenance or fighting other fires.
As for correcting others' errors, we don't always know about their errors (even if we ask). I think we should, though, correct errors in the public's perception that we're aware of, and we don't always need to point fingers to do that.
I am confused now from your response.
1. The checklist suggests we ask sources about errors in others reporting (a great idea I think). I assume these suggestions of errors by sources are then independently fact checked to determine if correct, and then included in our articles or why have this step in the checklist?
2. We encourage the public to contact media outlets to report errors. That outlet then reviews the facts and evidence they present and decides if they will correct or not, and if the correction will be transparent. We don't discourage the public by suggesting this is "finger pointing."
3. If it is healthy for the public to flag errors and to seek corrections, why would it be pointing fingers to transparently correct another media outlets errors in our own reporting? Why wouldn't this also be healthy?
4. The example I used: determining the correct number of fire trucks on a fire scene is knowable and verifiable with multiple sources of documentary evidence in my experience . Are you suggesting the number of fire trucks on a fire scene is not knowable with verification steps used in your checklist?
Rhonda, I'm replying here as a new comment, rather than replying directly, because the nesting makes the columns so narrow. But here are my answers: 1. You should definitely ask about errors in other reporting, so you avoid repeating errors. I don't think, thought, that you always have the obligation to correct others' errors. If an error is widely reported, you should correct it, but I don't feel a need to correct every detail anyone else got wrong. If I wouldn't include the detail in my story, I don't necessarily have to correct it.
2. I am not against identifying the sources of other errors. Neither do I feel a necessity to track down all other errors. If an important error was reported by another major outlet, I'd correct and say who made the error. If a lesser error was reported in multiple outlets, I wouldn't feel a need to identify them all, but just say it was initially reported incorrectly by other media.
3. I think both with the public and with the competition, some will see naming other media as throwing stones from a glass house. Our most important obligations are to ensure the accuracy of our own reports and to correct our own errors. If our readers/viewers have likely been exposed to important errors, we should correct them, too, but I'm more concerned with giving our readers/viewers the accurate information than with assigning blame.
4. I have covered fires where equipment arrived during the battle. So one number would be accurately reported early and another would be accurately reported later. Some might report the total vehicles, while others might report pumpers, ladders, rescue vehicles and district chiefs' trucks separately. For as huge a fire as you described, a reporter's access might be too limited to count all vehicles, especially with rescue vehicles coming and going. Some might, as you did, estimate the number of vehicles based on the number of units, but, as I noted, that's not always accurate. You should report accurately to the best of your ability and note if your figure is an estimate.
Thanks for the response.
1. Do you have any example where the public has criticized a news outlet for flagging errors in another news outlet's reporting? I don't know of any.
2. Should the public only ask for corrections if they are widely reported but not of details or lesser errors?
3. Is reporting who made an error assignment of blame or transparency so people know the source of the error?
4. Isn't the principle of correcting all errors the best approach as we can't always know or determine which error is small.
5. Regarding the firetruck count. I think you missed my point about best method. Instead of trying to rely just on counting vehicles yourself, the key for accuracy is for reporters is get the official account from the fire dept. or emergency response center as they dispatched and recorded in their computer time-stamped records which fire vehicles were sent to the "box." They can provide the official fire report for verification. It is also good too to speak to the fire officer in charge of the fire who is writing the fire report in addition to the PR rep if you are a careful reporter.
Rhonda,
I think journalists are responsible for the accuracy of our own work. My approach in covering a story is to get out front and stay in front, rather than looking back to repeat the facts (accurate or wrong) reported by other media. As noted on my checklist, I think an important question to ask is about errors made in earlier reports (whether by me, my colleagues or my competitors). And certainly, as I learn of facts I will be reporting that have been reported inaccurately, I should note the errors, whatever the source.
But I'm focused on advancing the story. I'm sure we could get in a circular discussion (I think we already have) about what needs to be corrected and the best way to correct. My focus, though, is making sure that all my work is accurate and that I correct my errors.
As for the vehicles, I think that's another circular discussion. The first reporter at the scene, who counts five firetrucks and reports that is accurate. But more keep arriving, and we may not keep the count updated. As you state, an official, accurate count may be available eventually. But don't count on the spokesperson at the scene having complete information. The person in charge might have that information, but might assign a PIO to handle the media, who may not have complete information yet.
The point is to say what you know and how you know it, and, if you're covering a breaking story, say that you will update.
Important for reporters on a fire scene to note. You don't have to wait in larger fire depts. Computer printouts that list all the responding companies (the vehicles) as they are dispatched (come and go) are continually available on scene as they are printed out from computers in fire officers cars for their managing of the fire. Try to make friends and ask for these print outs-- then you can report as of 5pm, X amount of trucks responded, according to official count ....
I agree with many of your points like making it clear what you know and how you know it. Very important! Thank you. Not sure how advancing the story is different than correcting facts that were previously reported. If I learned it wasn't a donuts shop as I had read in 3 papers but a church, the story is advanced for me both as a reporter and a news consumer.