AI-visibility measurement specification

A vendor-neutral specification for prompt-set construction, model identification, sampling, metric definitions, variance reporting, raw evidence, and version history.

Use the research specification · Use the correction protocol

This is a method for running and reporting a study. RankRift does not publish measured runs or datasets of its own; apply the method to your own work or use it to check someone else's.

In short

This specification describes what to count in an assistant answer; how to write, version, and freeze prompt sets; how to identify models and interfaces; how to schedule repeated samples; how to define metrics; and how to report uncertainty. It also defines raw-response retention, method versioning, and dispute records. Apply it to reader-run or external studies without inferring a RankRift dataset.

Who this is for

In-house SEO and organic growth leads who have to defend a new metric to a leadership team that has never seen one before
Agency and consultant strategists who need a method they can hand a client and have it survive scrutiny
Analytics and RevOps people checking whether a visibility figure is statistically meaningful or decorative
Founders and heads of marketing deciding whether any AI-visibility number deserves budget
Journalists, researchers, and skeptics who want to reproduce or refute a published result

When to use this

Before citing any visibility figure where someone might ask how it was produced
When you are writing your own measurement protocol and want a worked specification to adapt or argue with
When a vendor shows you a score and you need a checklist of what an honest method has to disclose
When a result looks wrong to you and you want to know exactly which step to interrogate

What the method measures

AI assistants do not return a ranked list of ten links. They return prose that names some brands, omits others, sometimes cites sources, and sometimes frames a brand as the wrong choice. So the unit of measurement is not a position. It is the content of an answer to a realistic buyer question, observed repeatedly.

Measure four things: whether a brand is named, whether and which source is cited, its share of named-brand space within the answer set, and how it is framed. Each answers a different question and must retain its denominator.

Other counts are downstream of those four. Do not collapse them into a composite score: one number hides which component moved and therefore hides what action the evidence supports.

How prompt sets are built, versioned, and frozen

A prompt set is the list of questions the study asks. Publish it in full with its version; a set that quietly favours a brand produces a flattering number that more sampling cannot fix.

Sets are written in the language a buyer would actually use, not the language a marketer would use. They span buyer stages, from open category questions through constrained questions to direct comparison questions. Each intent is expressed in several phrasings, because phrasing changes answers and a single wording is a measurement of that wording rather than of the intent.

Do not brand-bait. Prompts naming the subject brand invite the model to discuss it and inflate downstream metrics, so exclude them from headline figures or report them separately. Assign and freeze a version before a run; a changed set begins a visibly separate series.

A study without its full prompt set attached is an assertion, not a measurement.

How models are pinned, recorded, and scheduled

Assistants change underneath you. A model is updated, a retrieval layer is swapped, a system prompt is revised, and yesterday's answer becomes untrue without anyone telling you. If the model identifier is not recorded with the result, a later shift is indistinguishable from a change in your own visibility.

Every run records the model identifier and version string returned by the provider, parameters, interface, and call timestamp. Record whether the identifier is pinned or a floating alias whose underlying model may change.

Schedule runs rather than collecting them opportunistically. A fixed cadence supports comparison with a stable baseline, and spreading samples limits the effect of a transient provider condition. Treat a behavior claim as supportable only when it cites dated provider documentation or a reproducible recorded run.

How the metrics are defined

Mention rate is the share of sampled answers in which a brand is named at all. It is the crudest and most robust signal, and the one least sensitive to judgement calls. It answers "are we in the conversation." Citation share applies only to answers that carry citations. It is the share of cited sources attributable to a domain within the answer set. It answers a different question from mention rate: not whether the model names you, but whether your pages and the pages about you are the evidence the answer is built from.

Share of voice is a brand's named appearances as a proportion of all brand appearances across the same answer set. It is comparative by construction, so it is only meaningful when the competitor set and the prompt set are stated. Share of voice moves when a rival gains, even if you have not lost anything.

Framing is the classification of how a brand appears: recommended outright, listed among options, mentioned in passing, or named as a poor fit. This metric requires judgement, so apply a written rubric, sample agreement between named reviewers, and attach the rubric version. Report weak agreement rather than presenting the number bare.

Variance, confidence, and under-powered results

Assistant output is non-deterministic. Ask the same question twice and you can get two different brand lists. That is not a defect to be smoothed away, it is the property being measured, and it means a single answer is an anecdote rather than a data point.

Every reported figure therefore needs the number of runs behind it and an interval expressing how much it could move through repeat sampling alone. State the interval method rather than presenting a bare range. Report comparisons between brands, models, or dates as a difference with its own uncertainty, never as two point estimates left to imply a gap.

When a result is under-powered for the change it claims to detect, label it under-powered or omit the claim. The durable conclusion is “this sample cannot distinguish the brands at the stated threshold,” not a headline rounded from an ambiguous result.

Before publishing a dependent figure, record the interval method, assumptions, reviewer identity, review scope, and review date. If no qualified review occurred, say so and keep the claim within the calculation's demonstrated limits.

Data collection, storage, and reproducibility

Use official provider APIs and documented interfaces only. Do not automate consumer accounts, drive assistant web interfaces, or work around rate limits. A method built on an undocumented surface can break without warning, and a figure produced that way cannot be defended if challenged.

This choice has a consequence worth stating plainly: API output and the consumer product experience are related but not identical surfaces. Read results as measurements of the interface actually queried.

For a reproducible study, retain each raw response with its request, model identifier, timestamp, and prompt-set version. Derive metrics from that store rather than typing them by hand. If publishing a dataset, include the raw layer needed to recompute each headline figure, a licence, and a citation string; otherwise state the access limit explicitly.

Method versioning and what triggers a bump

Assign the method a version number and pair every figure with the version that produced it. Without that pairing, a reader cannot know whether an older and a newer number were made the same way.

A version bump is triggered by any change that could move a number for reasons unrelated to the thing being measured: a change to a metric definition or formula, a change to the framing rubric, a change to sampling or scheduling rules, a change to which interfaces are queried, or a change to how citations are attributed. Adding a category, adding a model to the roster, or fixing a typo does not bump the version, though roster changes are logged.

Record each change in a dated changelog with what changed, why, and which figures are affected. Update the method before publishing dependent content. If an implementation diverges from its stated method, correct either the implementation or the method and disclose the affected interval.

What this method cannot tell you

Being explicit about limits is part of the specification. These are questions the measurements do not answer, and no amount of sampling will make them answer.

If a result looks wrong, create a dispute record naming the study, figure, and suspected measurement error. Re-run under the recorded method and log the outcome either way. Payment or commercial preference must never change a measured result or its ordering.

It cannot tell you what any individual user saw. Personalization, session history, region and account settings all shape a real person's answer; the method measures a defined interface under stated conditions.
It cannot tell you why a figure moved. Detecting a change and explaining it are separate jobs, and model updates can remain an unresolved confounder.
It cannot establish that visibility caused revenue. Attribution is a different discipline with its own limits.
It cannot rate product quality. A measured visibility figure is a factual statement about model output, never a judgement about whether a brand is good.
It cannot promise that acting on a finding will change what any assistant says. A measurement gives nobody control over a model's output.
It cannot cover unqueried surfaces. Anything outside the recorded model and interface list is unmeasured, not implied.

A correction record must preserve the original figure, re-run inputs, outcome, and decision, whether or not the result changes.

Frequently asked questions

Why publish a method before choosing a tracker?

Because a number you cannot check is worth nothing. A public specification lets readers find flaws and constrains the operator: a method change must appear in the changelog before dependent content ships. The method remains usable whether the work is manual or performed with a vendor tool.

How should a study handle a model being updated in the middle of a measurement programme?

Record the model identifier and version returned by the provider on every call, so an update appears in the data rather than hiding in it. When a version changes between runs, annotate the comparison and report the pre-change and post-change periods separately. Do not smooth across a model change, because a figure that moves for that reason is not evidence of a visibility change.

Why does the method refuse a single overall visibility score?

A composite score compresses several independent signals into one number, and the compression is where the misleading happens. Mention rate, citation share, share of voice and framing can move in opposite directions in the same run, and knowing which one moved is the entire basis for deciding what to do. A single index also tempts everyone into comparing scores built from different prompt sets, which is meaningless.

What does it mean when you say a result is under-powered?

It means the number of runs behind the figure is too small to distinguish the claimed effect from ordinary run-to-run variation. In that situation report that the sample cannot distinguish the effect. Label an under-powered result only when its limits remain informative; otherwise omit the dependent claim.

Why only official APIs when the consumer interface is what buyers actually use?

Two reasons. Automating consumer interfaces can mean driving accounts or working around deliberate controls. A method built on an undocumented surface also breaks without warning, taking historical comparability with it. State the trade-off: figures describe the queried interfaces, not every surface a buyer might touch.

Can I reproduce a study that follows this method?

Yes; that is what the method is for. A reproducible study provides its full prompt set, model identifiers and versions, run dates, run count, metric definitions, and the data needed to recompute headline figures under a stated licence. A repeat need not be identical because the system is non-deterministic; compare it with the reported interval and document any difference.

What happens if I think a figure about my brand is wrong?

Identify the study, figure, and suspected measurement error in a dispute record. Settle the question by re-running under the recorded method, then log the dated outcome whether or not it changes the original. Payment must not alter a result, placement, ordering, or inclusion decision.

Does a low measured visibility figure mean my product is worse?

No. The figure describes how often a model named a brand under a stated set of questions on stated dates. Models reflect the text available about a category, shaped by coverage, category language, entity clarity, and source age as much as by product quality. Read a visibility figure as a statement about the sampled model output, not the market.

Check the method against the work

A specification only counts when a study follows it. Use the research record to check each required field and use the correction record when a figure does not hold up.

Use the research specification · Use the correction record