Methodology

Four Claims We Retracted, and Nine We Refused to Make

Four claims we corrected in public and nine we refused to make. What a substantiation standard looks like when it operates under load, receipts attached.

On 2026-07-25 we published a substantiation standard and pointed it at nobody but ourselves. Four weeks later, this is the dated record of what that standard removed.

That post — Receipts, Not Vibes: The Substantiation Standard — was forward-looking on purpose. It described a rule we intended to enforce on our own copy first and carried no confession, because at the time we had a rule and no record of it under load. This is the record: four claims retracted from client-visible surfaces in the four weeks since, nine stopped before they reached a document, and one impressive number that survived only after two words of copy changed around it. The argument has not moved. A substantiation standard is worth exactly what it removes, and the removals have to be visible, because a standard whose output nobody can see is indistinguishable from one nobody runs.

Everything below came out of the August monthly engagement ledger and its sibling build records, read 2026-08-20. Items with a date of their own show it; the rest sit inside that four-week window and nowhere more precise.

Receipt, Qualify, or Cut

Every claim on a surface a client or a stranger can read faces three doors, and no fourth one.

Receipt. The claim stands if someone can produce the artifact behind it on demand — a dated read, an API response, a capture, a named person saying the words. The receipt for a clean caller-reputation row is never the row itself; it is the known-bad number run beside it in the same session, because a provider quietly returning nothing looks exactly like one reporting a healthy line. Registration Is Not Reputation carries that August 2026 baseline in full, including the cells recorded as unknown rather than clean.

Qualify. The claim gets rewritten to what the evidence supports, usually by becoming less quantitative. The permitted claim on that same phone estate is exactly registered, monitored, labels re-checked monthly — never "spam is fixed." Registration and labelling are separate columns, the vendor's own text says registration does not guarantee positive treatment, and the sentence we hand the client says so: we fixed the configuration, the carriers take time to re-evaluate, and we will not claim the reputation outcome until we can show it.

Cut. The claim is deleted and the reason written down, so it cannot come back wearing different words. Most of this post's second half is the Cut door.

The enforcement is the part worth copying. The scan for unsupported claims runs automatically at the close of every work session, not at the discretion of whoever is still awake at the end of a long build. Vigilance is not a control. A control runs whether or not anyone remembers it.

The four we retracted

1. A voicemail backlog described as destroyed revenue. Overstated. Notification-by-email with attachments was enabled on three of the four boxes, so the deletion behaviour we reasoned from removes the in-app copy, not the message. The correction went onto the client-visible dashboard that carried the original claim, rather than being dropped quietly from the next version. Two standing instructions followed: on any voicemail-severity question, read the earlier finding first — one document's wording produced the same error twice — and do not build claims on in-app unread counts.

2. A client's numbers described as "unregistered." The wording was wrong and was corrected on the client-visible dashboard that carried it. We are not publishing the evidence that overturned it, because that evidence did not survive two independent corroboration passes — and an uncorroborated exoneration is the same failure as an uncorroborated accusation, pointed the other way. The correction stands alone: we said unregistered, we should not have, and the record now says so.

3. An English 911 instruction described as "proven." Reverted to unverified when the vendor setting its proof depended on was later disabled, and published as a regression rather than left standing as a pass. A claim whose evidence has been withdrawn does not stay proven because it was proven once.

4. A hardcoded gate date in the render layer. For days the client-facing view disagreed with the data seam underneath it. The dashboard was not lying about the numbers; it was lying about when — which is worse, because the figures looked fine and nothing invited a second look.

The refusals, which are better than the retractions

A retraction means a claim reached a reader. A refusal means it never did. Nine, same window.

A vendor concurrency figure. A capability number for a telephony vendor arrived through a search summary and was headed for a document. Fetching the cited source directly returned a page containing no such claim — a summarisation artifact with no first-party ceiling behind it anywhere. The instruction on record is short: do not let this into any document. Nothing to qualify, so it is a Cut.

An FCC rule vacated in 2025. Widely repeated, reads as current, no longer law. A vacated rule cited as live is worse than no citation, because it makes the paragraph around it sound researched.

A state compliance mandate the client is exempt from. The first draft treated it as an overstatement to be softened. It was not an overstatement — the exemption applies and the requirement does not attach. Softening a false claim is precisely how a false claim survives review.

A HIPAA framing that does not apply. The trigger is legal role, not product line — a business can sell something health-adjacent indefinitely without becoming a covered entity or a business associate. Removed rather than hedged.

A "zero key events" claim. A later live read of the analytics property returned three key events, all created before the engagement began, none modified since. The Receipt door working in the direction nobody enjoys: the artifact was produced, and it disproved us.

An API operation that returned 400. A configuration write we wanted returned a 400 and was reverted, rather than left in the runbook as a documented path that can never execute. The requirement was met another way — a site-level inheritance setting rather than hand-typed per-line writes — and read back after.

A registry filing halted unsent at the terms screen. The submission had been staged under the wrong entity's address, and was stopped at the last screen before certification rather than sent and amended later.

Three numbers held back from that filing. Of thirty-two lines, twenty-nine went in and three were excluded on purpose: two belong to a different entity, one is fax-only. Registering another entity's numbers would be a false certification — and on a filing whose entire product is a certification, that is the whole thing.

A red team that caught its own overclaim. The first draft of a disposition log — the document whose only job is to state what was fixed — claimed five fixes that had not actually been applied. A two-agent cold audit run after the plan was frozen caught all five, plus seven smaller errors. The log describes the verified final state only because that audit ran. That is the least comfortable line here and the most important: the machinery that catches other people's overclaims produced one, and you are reading about it only because a second pass was scheduled before anyone knew it would be needed.

Four retractions and nine refusals A two-part ledger. Retracted and corrected in public: a voicemail backlog framed as destroyed revenue, numbers described as unregistered, a 911 instruction described as proven, and a hardcoded gate date. Refused and never shipped: nine claims including a search-summarisation artifact, a vacated FCC rule, an inapplicable mandate, a filing halted unsent at the terms screen, and a precise weld count nobody had confirmed. The closing callout carries the 1,200-plus label change. RETRACTED — corrected in publicVoicemail backlog framed as destroyed revenue — overstatedA client's numbers described as “unregistered”An English 911 instruction described as “proven”A hardcoded gate date disagreeing with the data seamREFUSED — never shippedA vendor concurrency figure — a search-summarisation artifactA vacated FCC rule cited as live lawA compliance mandate the client is exempt fromA HIPAA framing that does not applyA “zero key events” claim a later live read contradictedAn API path that can never work — reverted, not left standingA filing halted unsent at the terms screen (wrong entity)Three numbers held back — registering them would be a false certificationA precise weld count nobody had confirmedThe closing pair: a precise animated counter shipped as “1,200+” — and “Welds YTD”became “Welds / Yr.” Two words of copy closed an implied-claim gap the number could not.
The substantiation standard operating under load, four weeks after it was published forward-looking. Every row traces to a dated ledger entry.

The counter that had to be qualified

A hero counter on a welding and fabrication shop's new site animated up to a precise, impressive weld count that nobody had ever checked with the owner; asked directly, he said more than 1,200, and it shipped as 1,200+ — a statement rather than a measurement, with no production log reconciled and no job records counted. The precise figure it used to show is not reprinted here, because printing it would re-publish the thing we removed. The subtler half was the label, which read Welds YTD and now reads Welds / Yr: on a page shipped in June a year-to-date total invites the reader to double it into a run rate nobody ever claimed, and two words of copy closed an implied-claim gap the number alone would not have.

That distinction is not ours. 16 CFR §255.2(a) governs consumer endorsements, not weld counters — we borrow it as a standard, not as the rule binding this page — but it states the principle cleanly: the advertiser must possess and rely upon adequate substantiation for express and implied claims, in the same manner it would if it had made the representation directly. The current Endorsement Guides were revised 2023-07-26. The implied half is what catches marketers, because implied claims are made by layout, label and tense — not by sentences anyone thinks to check.

We publish this record rather than fixing it quietly for a reason that is commercial as well as ethical. Forrester's 2026 prediction that trust becomes the ultimate currency for B2B buyers is a bet that verifiable claims out-convert louder ones. We take that bet in the only form that costs anything — publishing the corrections.

What would prove this wrong

The claim here is narrow: a substantiation standard applied to our own copy produces visible removals, and the enforcement is automated rather than aspirational. Four ways to break it.

  1. Demand ten receipts. Pick ten numeric claims live on our site or on a client dashboard and ask for the artifact behind each. One that cannot produce a dated read, a capture, or a named person's statement inside a working day falsifies the standard as a control.
  2. Check where the corrections live. Every retraction above was corrected on the surface that carried the original claim. Find one silently deleted instead and the "visible" half of the argument fails.
  3. Ask for the denominator. Four retractions and nine refusals is a numerator. A rate needs the total claims made in the same window, and we do not publish one, because we have no defensible way yet to count claims across every surface. Until we do, this is a list, not a rate.
  4. Notice who caught these. Every item was caught internally; none by a client. That is the outcome we want, and also exactly what a standard quietly missing an entire category would look like from the inside.

The honest limit is the third. The record shows the standard operating. It does not show what the standard missed.

Run the Receipt door on yourself first

List every number currently live on your own website, and write beside each one the artifact that would satisfy a stranger who asked for it today. The blanks are the shortlist, and the length of the blank column is the finding. Where a second reader would help, we will work the list with you and mark which of the three doors each claim belongs in.

  • #blog
  • #substantiation
  • #corrections
  • #transparency
  • #compliance
  • #honest-marketing
Share X LinkedIn
Build it yourself?

Get the kit, not just the theory.

We'll send the build checklist behind this post — and the next pillar when it ships. One email, no drip sequence. Unsubscribe in one click.

Want this built for you?

Book a discovery call. We'll walk your numbers.

20 minutes. Tell us what's broken, hear what we'd ship in the next 90 days. No pitch deck.