Innovations in Media: 'Primary' — not 'Objection'

I started out here with plans to highlight a new startup called Objection. But in a world of pivots, before I could start writing, Objection” has transformed itself into a new venture called Primary. Regardless, both ventures are worth exploring. 

The June 2021 Hollywood Reporter article that spawned a complaint via the Objection website.Let’s start with Objection.ai, which I first learned of from a June 10 article in The Hollywood Reporter. That article — headlined “An Online Tribunal Funded by Peter Thiel is Putting Journalists on Trial. “I’m Case No. 1” — certainly caught my eye. The article is well worth reading, but here’s the gist: 

Anyone who is the subject of a news article can “object” to anything within that article by submitting a complaint to Objection. The site then contacts the author of the article, inviting them to defend their work by uploading supporting evidence. Objection then would send the information to a “tribunal” of artificial intelligence “judges” to gauge the legitimacy of the journalism. The AI judges included publicly available models like Claude, ChatGPT and Grok. 

Objection charged complainants $2,000 to $15,000 for each filing to cover investigative costs. 

According to Spear’s magazine — described as a “wealth management and luxury lifestyle media brand whose flagship magazine has become a must-read for the ultra-high-net-worth (UHNW) community” —  Objection’s models were “supervised by a special purpose judicial AI.” 

“Five LLMs each deliver a verdict, choosing from one of three options based on the balance of probabilities: ‘true’, ‘false’, or ‘insufficient evidence’. The votes are tallied by Objection’s protocol, which delivers a consensus verdict that is made public along with all the documentation and an explanation of the decision.”

Objection would also issue an “Honor Index score” to the reporter/writer/publication in question. For an extra fee, the complainant could then pay to have the findings promoted to the wider Internet. 

Yikes. 

Among Objection’s funders is/was Peter Thiel, the billionaire who bankrolled the prominent libel lawsuit that forced the infamous Gawker website into bankruptcy in 2016. Objection founder Aron D’Souza — who helped Thiel in his Gawker lawsuit — launched the venture as a way to counter what he believes is a flawed system to foster journalism accountability. 

At the time of the Spear’s article, Objection had issued public adjudications in two cases filed by the company itself to highlight the review process. Those two cases investigated allegations against podcaster Joe Rogan and French First Lady Brigitte Macron. Both allegations were found to be false. 

After reading the Hollywood Reporter article, I checked the Objection.ai site only to read this message: “Objection is now Primary,” followed by this explanation: 

“We built Objection for the moment after damage was done. If the media got facts wrong about you, you could file an objection. We'd launch an independent investigation, and an AI Tribunal would issue a verdict on the merits of evidence submitted for and against statements made.

It worked. A little too well…

What we didn't expect was that every case would point to the same root cause:

Bad reporting isn't a journalist's problem. It's an incentive and editorial problem. Modern journalistic economics run on clicks and advertising — and clicks reward speed, volume, and outrage. The quality of the reporting itself has never had a scoreboard or feedback loop. There has never been a way to measure the discipline and rigor behind a story, so there has never been a way to reward it. Journalists know what great work looks like. The system just stopped paying for it.

Verdicts punish failure. They don't fix the incentive. They do not solve for the root cause.”

So we built what is missing: a robust public methodology that scores, ranks and indexes journalists on the rigor of their reporting. Every journalist, on the record, continuously.

When the work is measured, the work gets rewarded — and journalists get back to doing what they do best: original, rigorous reporting.

Objection issued verdicts. The Primary keeps score.”

Hmmm. 

An initial visit to the Primary website can be overwhelming, but after a few minutes, I found myself absorbed by the depth and breadth of the individual article, publication or journalist pages. Primary currently has individual pages for hundreds of journalists from a dozen U.S. and British outlets, including Reuters, The Guardian, The New York Times, Fox News, CBS News, Daily Mail, The Telegraph and The Washington Post. 

A Publication index page highlights top-level metrics for individual news outlets.

Rankings are determined by a wide swath of variables, including Sources, Evidence Quality, Evidence Traceability, Balance and Uncertainty, Fact Vs. Opinion and Polemical Language (the latter is defined as “How free the writing is of emotionally charged, loaded, or one-sided language. Higher means more neutral, measured wording. Polemical Language is an experimental metric which may contain errors.”). Graphics plot trends and performance over time, with links to detail pages for specific articles or reporters. Each journalist is given a  “political leaning” ranking ranging from left, center and right. Journalists can claim their own profile pages, and submit complaints to a “review team” if they question any findings.  

Detail pages for individual journalistsm can be mesmerizing, with entry points to individual pages where specific articles are analyzed.An analysis of a New York Times article written by Lisa Friedman, a former co-worker at The Bakersfield Californian.

It’s difficult to make sweeping judgments in reviewing the data, but I found it interesting that broadcast networks — including NBC News, CBS News, ABC News and Fox News — generally scored higher than newspapers. NBC News ranked No. 1 among news outlets, followed by The Washington Post and CBS News. The Daily Mail was by far the worst-scoring of the 12 news outlets analyzed. 

The Primary is certainly an improvement over the heavy-handedness Objection seemed to pursue. The new site has ingested several million news articles, so at least there’s some depth to the analysis. I suspect the methodology will evolve over time, which may affect trending analysis.

I worry about assigning scores to quality journalism, which often involves nuance when factors don’t fit predefined quality metrics. But I’m all for fair, accurate and responsible tools that can help an overwhelmed public sift through the daily onslaught of news, propaganda and outright lies that flood our feeds. Let’s hope the Primary lives up to its goals of rewarding good journalism rather than Objection’s misguided mission to attack good journalism.