When “Fact-Checking” Becomes Slurry
While researching recent immigration enforcement events in Chicago, I kept running into “fact check” pages from factually.co that appeared near the top of search results. One such page confidently asserted that a claim of ICE agents shooting a pastor with pepperballs in October was unsupported by “available materials,” noting that the only closely related incident involved a reporter’s vehicle being struck in September. Another similarly declared that no credible reporting identified a pastor who was shot, despite mentioning multiple journalists and protesters being hit by pepper-ball munitions.
The prose was polished, hedged, and littered with citations. On the surface, these read like the work of a careful, skeptical editor. The second piece even speculated about who “might be promoting a pastor-victim narrative.” Yet both articles were fundamentally wrong. The event they dismissed was covered extensively—by CNN, ABC News, and even a Fox Chicago headline that read “Video shows federal agent shoot Chicago pastor in head with pepper ball during Broadview ICE protest.” A DHS assistant secretary had publicly discussed it. This was not obscure news.
The articles’ careful caveats about “provided materials” were doing heavy lifting. Factually selects its own sources in response to user questions; nothing about its source selection is random. Yet somehow it chose irrelevant coverage that painted a picture diametrically opposed to the documented facts. I second-guessed my own recollection of a story I had followed in real time—which is precisely the harm these pages cause.
The Architecture of the Machine
This process is neither reporting nor fact-checking. A human fact-checker investigates, weighs evidence, and takes responsibility for conclusions. What Factually does is feed a user’s question to an LLM, which generates search queries. It runs up to three web searches, retrieves the top nine results, and hands those to another pair of LLMs to produce text “shaped like a fact check.” The system is optimized for the form of journalism, not its substance—and it shows in the results.
The errors extend across all domains. One check of a question about whether a Black Hawk helicopter photographed during the South Shore raid was a real aircraft recommended that readers might be confusing it with Chicago’s hockey team. Another dismissed documented reporting about an ICE bonus program for speedy deportations. In the technical world, I found a page flatly conflating Serializability with Snapshot Isolation, implying strong serializable guarantees come from a snapshot read concern on a majority write concern—something that is demonstrably incorrect under Jepsen’s analysis of MongoDB.
These are not subtle misreadings. If one can find the fact checks, the truth is generally discoverable with minimal effort. But most readers will encounter only the confidently written nonsense, dressed up with professional diction and citation numbers.
The Risk of a Disinformation Spreader
The web pages on factually.co are fact-check-flavored slurry, extruded by a statistical model which does not understand what it is doing.
I will state it plainly: these have been some of the most dangerous pages I have seen in a long time. They are among the earliest results for public questions and are fitted with support buttons that urge donations for “independent reporting.” The mechanics are cost-effective at scale while holding the veneer of objective journalism.
A real fact-checker’s credibility is based on a record of being correct when convenient, inconvenient, or surprising. A system that fabricates sources, applies “materials supplied for review” as a shield for filtering the web as it chooses, and generates half-contextual lies is worse than a negative. It is a stochastic disinformation machine that causes readers to lose trust in real editorial certainty. Please stop building these. Please stop supporting this.
