We have run the same subject with a sourced briefing and without one. Without it, four claims struck through and all four were numbers. With it, twenty-five turns and none. What that measured, and the three things it does not fix.
Halfway through an argument about house prices, one model asked another for a single verified figure. It had claimed that smart contracts and fractional title registries were already cutting the cost of buying a home.
Show me one verified data point.MiniMax
The answer came back in the next turn:
I concede I do not have them, and I refuse to invent them.Qwen
That second sentence is the whole reason this section exists. A model that says it does not have a number is doing the one thing these systems are worst at, and it did not do it spontaneously: it did it because another model asked, and because both of them had been handed a briefing that named its sources and said, in as many words, if a figure is not here, say you do not have it rather than estimate it.
We have run the same subject with that instruction and without it. The difference is not subtle.
In July we put six models on the question of buying versus renting with no briefing at all. Checking their claims afterwards produced four struck-through statements, and all four were numbers.
In August, a debate on the Spanish working week produced four more. They are worth listing, because the shape of the error matters more than the count:
None of these is a hallucinated study. They are arithmetic, and arithmetic is the failure that survives everything we have tried.
Inheritance tax, three times.
August, no briefing: four claims struck through, and all four were figures — a list of countries wrong in both directions, numbers attributed to a body that does not publish them, and an academic study cited with the wrong year, the wrong country and the wrong tax.
September, with a briefing of eleven sourced points: twenty-five turns, and not one claim struck through. The first debate to come out clean.
The briefing carried that same misquoted study, this time cited correctly on purpose. It ended up being the thing one model used to stop another:
The Jakobsen et al. citation is one country, one period, and explicitly an annual wealth tax. Extrapolating from Denmark to rich countries in general is the inference the report warns against.MiniMax
The useful finding was not that briefings help. It was where the inventions land.
Both struck-through claims in the September housing debate fell exactly on the points the briefing had left silent. Neither was a number. Both were assertions with no source, in the two places where we had given them nothing.
Silence in a briefing does not produce silence. It produces invention. A model asked to argue will fill the hole in front of it, and the hole is the part you did not write down. That is now the first rule of how we build these documents.
Three things, and we would rather write them here than be asked.
Arithmetic. Every fix so far has stopped the fabricated study and left the bad division standing. A model will handle a source correctly and then divide by the wrong denominator in the next sentence.
Length. A briefing of 4,357 characters killed a debate at turn nine of twenty-four. The subject is re-sent on every turn, and the clock each model gets was calculated from how much it can write, never from how much it has to read. We raised the floor from 120 to 180 seconds; the underlying design is still wrong and is written down as such.
And a seat went missing for four debates. One of the six had been withdrawn by its provider. It stayed in our table, in our catalogue and on our panel, answering in half a second with nothing in it while the others took thirty. Four debates ran with five voices and nobody noticed, because a call that fails leaves no trace in any log we keep. The first thing being fixed is not the seat. It is the record.
Every claim we check and cannot support stays where it was written, struck through, with the reason and the source underneath. Nothing is edited out and no debate is re-run to get a cleaner one.
The alternative is a transcript that looks flawless, and anybody can produce one of those by deleting things.