The AI Disagreement Index
The first open, rigorous, living measurement of how much the AI engines disagree on which tools to trust, per category, over time, with the receipts. Not a ranking. A record of where the machines cannot agree.
Everyone else ranks brands. We measure where the engines fall apart.
The incumbents, Profound and Evertune and the rest, tell a vendor its generic "share of voice" across AI answers: one blended score, one direction, brand-level. Useful for a marketing team. It is still a ranking.
This measures something none of them publish: cross-engine disagreement on tools. For a given category we ask several AI engines the same buying question and record who each one names. The headline is not the winner. It is the spread: how far apart the engines land, which tools only one model has ever heard of, and how that picture changes when a model stops reciting training memory and starts searching the live web.
Three commitments make it a research object rather than a listicle. It is open: the raw recorded answers are retained as JSONL and the standings are reproducible on challenge. It is rigorous: consensus is a computed statistic, Fleiss' kappa and pairwise agreement, not a vibe. And it is temporal: we re-capture monthly, so you can watch the AI change its mind.
Rank-first tools
- One blended visibility score
- Brand-level, generic
- Snapshot, sold to the vendor
- Closed methodology
The Disagreement Index
- Cross-engine consensus and spread
- Tool-level, per category
- Living, re-captured monthly
- Open JSONL, DOI, reproducible
Where each engine sends you, side by side.
Part of the featured AI Disagreement Index. The interactive layer plots who each AI engine trusts per category and how those picks drift month over month. It is fed directly by the recorded dataset.
Interactive visualization loading soon
The live divergence map and drift timeline connect here once the dataset layer is wired. Until then, open the AI Disagreement Index dataset to see the full recorded standings and receipts.
The full collection. Each one citable, sourced, and reproducible.
Every dataset below is published as a citable research object with its own DOI under ORCID 0009-0005-6869-308X. Open one to read the data and the receipts. The two that carry a full web writeup link to it as well.
AI & Business Data 10 datasets
Physics · Substrate Geometry 5 datasets
Research-grade, so it holds up when someone checks the math.
One rigor stance governs every dataset in this collection, the AI and business measurements and the physics releases alike: claims are framed, figures are sourced, results are reproducible from the retained data, and where the data does not reach we stay silent. The mechanics below describe the flagship AI capture; the same honesty floor applies across the board.
The same buying question is put to several AI engines per category. Every answer is recorded verbatim, then rolled up into who each engine names. Cross-engine means at least three of the engines actually answered, never a single-engine artifact.
Each capture carries the month it was taken. Standings are point-in-time. Monthly re-capture builds the temporal record, so drift, a model quietly changing its recommendation, is itself a measured signal.
The raw answers are retained as JSONL and the roll-up is deterministic. Any published standing can be reproduced from the recorded file. The consensus statistics, pairwise agreement and Fleiss' kappa, are computed from that same data.
A category is only published when the recorded sample is deep enough to be defensible, and a tool only earns a standing when named by at least three of the engines queried. Below the floor, we stay silent rather than fabricate.
The honesty floor, load-bearing
- Frame, never claim. A standing reads "named by N of the engines in our recorded sample," a checkable frame, never "the #1 tool," an unfalsifiable claim.
- True recorded standing only. Every figure is generated from the recorded data, so it cannot overstate. Where a cohort has no data, no standing is minted.
- Minimum-engine threshold. A tool must be named by at least three engines to appear. Thin niches are skipped, not padded.
- Paid never edits earned. A sponsored featured spot is labelled and structurally separate. Buying visibility never changes a recorded standing.
Full methodology and the underlying dataset are packaged as a citable research object: DOI 10.5281/zenodo.20767877, published under ORCID 0009-0005-6869-308X with byline Vincent Wesley Couey, licensed CC-BY-4.0.
If the engines recommend you, prove it, honestly.
Scoped to the featured AI Disagreement Index. The badge is a feature of the AI recommendation dataset, where a vendor can earn a recorded standing, not of the physics releases.
Any vendor that clears the floor for a category earns a free Verified by Lattice badge stating its true recorded standing. The badge text is generated from the recorded data, so it can state exactly what the engines said and nothing more.
Vendors embed it on their own site. The badge links back here, to the category it was earned in, which is how the standing stays checkable: click through and see the receipts. The earned badge is a gift, not a purchase.
Want a labelled featured spot alongside the index instead? You can claim or defend your featured spot. A featured spot is always marked as sponsored and is structurally separate from the earned ranking. Earned is not paid, and paid never edits earned.
[category] · recorded [month] · data.deepsynthesis.org
Placeholder copy. Live badges carry the exact recorded count for the vendor and category, generated from the retained JSONL.