Remix: Lessons Learned in the Launching of a Student Fact-Checking Unit

My heart skipped a beat and I swallowed hard at the off-handed, but characteristically direct, advice I received from the Wisconsin Center for Investigative Journalism’s Managing Editor as she pored over a student’s initial draft of a fact-check of a Wisconsin state senator’s claim about education policy.

“The last thing you want,” Dee Hall said, peering over her glasses, “is to start a fact-checking website and have to retract one of the first things you post because you made a fact error.”

Yeah. “Good tip,” I weakly replied.

Creating The Observatory with Lucas Graves has been a grand adventure. Even though we’re just starting what we hope will be a lengthy and fruitful collaboration, we have learned some things about birthing a student-led fact-checking team from nothing.

Innovate From the Classroom Out

While we hope that The Observatory continues producing great content during semesters and summers when one of us is not teaching the course, having regular meetings and a stable of students who have to be there – producing fact-checks for course credit – has proven crucial to ensuring we have enough content to launch our site.

Students are assigned to cover a competitive legislative race in Wisconsin and work in teams to cover competitive U.S. Senate races and the White House contest. In the future, we will expand our reach to all Wisconsin state lawmakers and pay less attention to national politicians who already receive scrutiny from fact-checkers across the nation.

My teaching assistant, Amelia Rufer, and I create a classroom space where our students pitch facts to check, do research, call sources, and tolerate us roving around the classroom-as-newsroom, having short, direct and sometimes demanding conversations about thorny issues in students’ work.

In addition to producing fact-checks, students write “community-based fact-check follow-ups” – stories that give a voice to the citizens affected by whether the claims we check are true or not. For example, one student produced an audio story interviewing constituents of a lawmaker who denied engaging in “pay for play” regarding donations made to a “dark money” group that aired ads on the candidate’s behalf, shortly before votes to help the donor’s businesses (rating: unobservable). After fact-checking a claim that Wisconsin’s roads are among the worst in the nation (rating: verified), another student made a by-the-numbers graphic of Gov. Scott Walker’s transportation proposals.

Because we are a class and can set inviolate due dates, we can publish the follow-ups concurrently with fact-checks, giving our audience a more comprehensive picture of the consequences of a claim’s veracity. Because the follow-ups have multimedia requirements (audio, video or data visualization), students build skills and give readers more reasons to stick with our site.

This would be more difficult in a start-up environment, where we had to rely on volunteers to produce stories whenever they can instead of whenever we tell them to.

Marry Well

We could not do The Observatory alone. Our major partners at WCIJ have been essential collaborators on a variety of fronts. From talking through logical inconsistencies with the color scheme of an early draft of the “Veritas Scale” (“veritas” is Latin for truth) we use to rate our fact-checks to web navigation issues to the long, frustrating and absolutely critical editorial process WCIJ goes through with every story they publish. (See Andy Hall’s post in this series that describes this amazing process.)

Working with a professional reporting organization has forced us to confront many issues we might have otherwise put off for too long or outright ignored. Because of firm, yet politely asked, questions, we will show our readers the process fact-checks go through before being published. Thanks to WCIJ, we have a set of professional photos of our team. We have a clearly defined editorial process. We have a professionally designed website.

More than anything, we have help. Outstanding help.

How to Pick Facts to Check

Current WCIJ intern and former Milwaukee Journal Sentinel fact-checking unit intern Cara Lombardo created a short guide for our students that covers choosing a statement to check, checking it and writing it up.

By far, the most difficult thing for our students is picking facts to check. Student journalists, like their more grizzled professional counterparts on the campaign trail, are drawn to newsworthy claims candidates make. But statements like “my opponent’s ideas are the worst our nation has ever seen” are subjective and not precisely checkable.

Students practiced by watching a Sunday morning talk show and selecting a checkable claim and describing the claim and how to check it to their classmates. I often begin class by listing claims on the board that students rank in order of “checkability.”

In general, we want students to choose claims that, as Lombardo writes, are:

• Numeric (“Median wages dropped more than 20% since my opponent took office.”)
• Comparisons (“Only three counties in the state have no public ice-skating rinks.”)
• Biographical (“I’m the one state senator who has actually worked on a dairy farm.”)
• Paraphrasing (“My opponent has said she thinks health care workers are overpaid.”)
• Interpretations (“If we pass this bill, our children will no longer learn cursive in school.”)

Even so, we feel that there is value in trying to show our readers what it would take to check a statement that is not precisely checkable. While most fact-checking outlets either use a scale ranging from true to false or eschewing a label for a holistic rating, The Observatory has a category called “unobservable.” When a claim is unobservable, we present as much information as we can to provide scrutiny to the claim, but we also clearly note why the claim cannot be strictly checked.

Make a Fact-Check, Check it (more than) Twice

Teaching students who have only taken one traditional reporting course presents a series of challenges: lack of proficiency with the AP Stylebook, little experience calling sources for interviews, and slight familiarity with the bread and butter of fact-checking: government documents. These challenges have led us to a five-layer editorial process before publishing:

• Style, logic, writing comments of the original draft from the TA
• Style, logic, writing comments of the original draft from the professor
• TA and professor raise potential issues of fact with the revised draft.
• Student responds to potential issues of fact with the revised draft
• Student goes through WCIJ’s fact-checking process

Expect More, Get More

Despite the general lack of experience most students have (one reporting course and experience writing for a school newspaper), students raise their game when the stakes are high. I’ve been teaching college students for 15 years, and I have never had a group so committed to doing their best work, behaving like professionals, wanting to get it right and taking tough criticism without complaining.

Nothing replaces the one-on-one relationships with students we develop in the newsroom environment we have created for The Observatory. Student work is maturing at a rapid rate.

This is a choice. Students could choose to fold when facing down the fact-check from an award winning reporter and editor at WCIJ. They could choose to phone it in after being told that their third draft needs revision because the fourth paragraph conflates proposed budgets and final budgets.

We provided a structure that allows students to produce fact-checks of claims local and national lawmakers and candidates for office claim are true. If students weren’t convinced that they would be doing something unique and of value, we would surely fail.

Our students are paying us to be hard on them. They could choose to quit. But they don’t. They come back working harder and smarter. And that, ultimately, is why I think we’ll succeed.

Michael W. Wagner is Associate Professor and Louis A. Maier Faculty Development Fellow in the School of Journalism and Mass Communication at the University of Wisconsin-Madison. His research, teaching and service are centered on questions relating to how elements of the information environment—e.g., the way politicians frame issues and the level of polarization in Congress—interact with individual-level factors to affect people’s political preferences, partisanship, and behaviors. He blogs at prowag.me and tweets @prowag.