AI marketing claims. The evidence, the limits, and what I'd test next.

Methodology

How we decide if something actually worked.

Three labels, never mixed

Every piece carries one label at the top. An experiment has a hypothesis, a control or baseline, a sample and a verdict. An observation is a pattern in real data without a control. An opinion is what I think, and why.

Four formats

Experiments test one claim. Field notes are shorter observations and opinions from the work, including analysis of external research. Playbooks are guides and workflows you can put to work. Benchmarks publish reference numbers from aggregated data, with the sample and period. All of them live under Research.

How an experiment runs

We write the hypothesis and the success metric before we look at results. We state the sample, the period and what we held constant. If the result is not significant, we say so — "we don't know yet" is a valid verdict.

What a result can and can't prove

Only a test with a proper control or baseline can support a claim about cause and effect. Without one, a result is an observation: useful, but not proof, and labelled that way. Every piece keeps three things apart: what we measured, what we observed and how we interpret it.

When a piece analyses someone else's study or report, it's filed as a field note, not as an experiment, and the original source is named and linked.

Where the data comes from

Our own tests and, in some cases, anonymized and aggregated data measured with Sealmetrics. No personal data is published. The source of every dataset is named on the piece.

What's free and what needs an email

The question, hypothesis, method and dataset of every piece are public. Some pieces put the detailed results behind a free subscription. Search engines and people see exactly the same thing: we never show crawlers content that readers can't see.

Independence

Unmodeled is written by Rafa, founder of Sealmetrics. Sealmetrics may provide measurement infrastructure; it does not choose topics or verdicts. Every page says who writes it: Rafa, founder of Sealmetrics. When a piece was measured with Sealmetrics, it says so.

Corrections

When we get something wrong, we fix it, update the "last updated" date and log the change at the bottom of the piece.

AI marketing claims,
examined.

Get the evidence, my take and what I’d test next.

Free. One-click unsubscribe.