Most teams start measuring AI visibility by running some prompts, writing down a number, and moving on. Six months later somebody asks what that number actually was, and the honest answer turns out to be nobody knows, because the prompts were edited, the models were updated, and the spreadsheet has no dates in it.
The number is not the deliverable. The record of how the number was produced is the deliverable, and it is the only part you cannot recreate later.
What a baseline actually is
A baseline is a measurement plus a complete description of the conditions that produced it, taken before you change anything, and preserved in a form a stranger could re-run.
Two things follow from that definition, and both are unwelcome.
The first is that a baseline cannot be reconstructed after the fact. If you ran prompts in March and did not record which prompts, on which models, on which dates, with how many runs, then in September you do not have a March baseline. You have a March anecdote. Re-running the same prompts today against today's models produces a September measurement, not a retrospective March one, because the system being measured is not the same system. There is no reconstruction procedure. This is the single most expensive mistake in the category, and it is entirely preventable with an hour of note-taking on day one.
The second is that a single run is not a baseline. Assistant outputs vary between identical calls, so one pass over your prompt set gives you a point drawn from a distribution you have not characterised. A baseline needs enough runs to say something about the spread, and it needs to be repeated at least once before you accept it, so that you know how much your "starting point" moves when nothing has changed. Treat your first two or three collection passes as establishing the baseline together, not as a baseline plus early results.
The provenance record
This is the artifact. Everything else in a measurement programme depends on it. Write it as a versioned document that sits next to the data, and fill in every field even when a field feels obvious today.
| Field | What to record | Why it matters six months from now |
|---|---|---|
| Prompt-set version | Identifier plus the full text of every prompt as run | Prompts drift constantly. Without a version, you cannot tell whether a change in the number came from the market or from an edit somebody made in April |
| Prompt-set rationale | Why these prompts, which buyer stages they cover, what you deliberately excluded | Future-you will be asked why a particular question is missing, and "we forgot" is a bad answer |
| Models and versions | The exact model identifier and version string for each model, per run | Providers update models. A version string is the only way to attribute a step change to a provider release rather than to your work |
| Access method | Which official API, which endpoint, which parameters including temperature and any system prompt | Parameters change results. An undocumented parameter change is indistinguishable from a real movement |
| Run dates and times | Date and time of every run, with timezone | Lets you line up a movement against a provider changelog entry or your own release |
| Runs per cell | Number of runs per prompt, per model | Determines whether any delta you later report is distinguishable from noise |
| Region and locale | Where requests originated, what locale settings were used | Answers can differ by region. If you later change infrastructure, you need to know what changed |
| Entity definitions | What counts as a mention of your brand and of each competitor, including variants and exclusions | Counting rules are where silent bias lives, especially for common-word brand names |
| Metric definitions | The formula for every metric, written out | "Share of voice" means four different things. Write yours down before anyone argues about it |
| Method version | A version number for the whole procedure, plus a changelog | The hinge that lets you change your method later without destroying comparability |
| Raw response storage | Where the full responses are stored, and the retention policy | Explained below, and non-negotiable |
| Owner | Who ran it and who can answer questions about it | People leave. Undocumented measurements leave with them |
Each field earns its place by answering a question you will actually be asked. When the number moves in November, the only way to distinguish "our work changed something", "the provider shipped a new model", "someone edited the prompt set", and "this is normal variation" is to have recorded all four possibilities in advance.
If you have not built the prompt set yet, do that first, because the provenance record wraps around it. Our guide to building a prompt set covers construction and the traps to avoid.
Storing raw responses, and why
Store the complete text of every response, not just your parsed metric. Store it alongside the prompt, model version, timestamp, and run identifier.
Three reasons, in order of how often they bite:
- Your counting rule will change. You will discover that your mention rule missed a shortened brand form, or double-counted a URL. With raw responses you re-parse the archive and get a corrected historical series. Without them, you can only apply the fix going forward, which leaves a permanent seam in your data.
- You will need to show your working. The first time an executive or a named competitor questions a result, the answer that ends the conversation is the actual response text. A parsed count invites argument, a transcript does not.
- New questions arrive later. Somebody will ask which sources were cited, or whether the mention was a recommendation or a passing reference, or what tone the model used. Those questions are answerable from an archive and unanswerable from a column of numbers.
Set a retention policy at the same time, write down where the data lives, and check that storing it is consistent with your own privacy commitments and your provider's terms before you start collecting. Storage volume is not the constraint people expect it to be; text is small.
Choosing your primary metric before you see the data
Pick one primary metric and write it down, with its formula, before your first run completes. Pick two or three secondary metrics and mark them explicitly as secondary.
The reason for the ordering is uncomfortable but well understood: if you choose the headline metric after seeing which metric flatters you, you have not measured anything, you have selected. It happens without any intent to deceive. You look at five numbers, four are flat, one moved, and the one that moved becomes the story.
Practical rules:
- The primary metric should be the one that matches the decision you are trying to inform, not the one that is easiest to move.
- Write the formula out in full, including how ties, partial matches, and non-answers are handled.
- Decide in advance how you will treat responses that decline to recommend anything, because they are common enough to shift a rate.
- Once chosen, the primary metric changes only through a method-version bump with a written reason.
Writing down what would count as success
Before the first result, write a short success statement and put it in the provenance document. It should contain four things:
- The metric and the direction. Which number, moving which way.
- A magnitude worth acting on. The size of change that would matter to the business, stated before you know what size of change is achievable.
- A timeframe. When you will check, chosen with reference to how long changes plausibly take to show up rather than to your reporting calendar. Our post on how long change takes gives the honest ranges and the uncertainty attached to them.
- What would make you abandon the approach. The result that would tell you this lever does not work, so you stop rather than rationalise.
The fourth item is the one people skip and the one that gives the document its power. A programme with a written stopping condition can be defended in a budget conversation. A programme without one produces a rolling explanation for why the next quarter will be different.
Be conservative with the magnitude. AI visibility movements are frequently within run-to-run variation, and a success threshold set below your own noise floor means you will declare victory on nothing. If you do not yet know your noise floor, that is another reason the baseline needs more than one collection pass before you commit to a threshold.
The re-run schedule
Decide the cadence at baseline time and hold it. Two rules keep it useful.
- Fixed intervals, not event-triggered. If you only re-measure after shipping something, you have no measurements of what happens when you ship nothing, which is exactly the comparison you need.
- Same protocol every time. Same prompt set version, same runs per cell, same models where they still exist, same parsing. Changes go through the method version, not through the run.
Monthly suits most teams. Weekly is usually noise for anything other than a fast-moving incident. Quarterly is defensible if your prompt volume is small and your budget is smaller, and it is a legitimate place to sit; the maturity model sets out what each level of investment genuinely costs.
Also schedule a provenance review. Once a quarter, someone re-reads the document and confirms it still describes what is actually happening, because drift between the documented method and the running method is the normal state of any measurement programme left unattended.
When the method has to change mid-programme
It will. A model gets deprecated, a provider changes an endpoint, your counting rule turns out to be wrong, or your prompt set no longer reflects how buyers ask. Changing the method is fine. Changing it silently destroys the series.
The procedure:
- Bump the method version and write a changelog entry with the date, the change, and the reason.
- Run both methods in parallel for at least one full cycle, so you have the same period measured the old way and the new way. This overlap is what lets you keep any comparison across the seam.
- Report the seam on every chart that spans it. A vertical marker and a one-line note. Never present a pre-change and post-change number as a clean delta without saying that the method changed.
- Re-parse the archive where possible. If the change is in the counting rule rather than the collection, your stored raw responses let you recompute history under the new rule and keep one consistent series. Say clearly which series a chart uses.
- Never backfill an estimate. If a model was retired and you cannot re-run it, the series ends there. Show it ending. An interpolated value is a fabricated measurement.
Model retirement deserves particular care, because it removes a data source permanently and no amount of process recovers it. When a provider announces a deprecation, capture a final run under the documented protocol before the endpoint goes away, and record that this is the terminal observation.
In Plain English
A baseline is not a number, it is a number plus the full record of how it was produced: prompt-set version, model versions, access method and parameters, run dates, runs per cell, region, entity and metric definitions, method version, raw response storage, and an owner. Write that record before the first run, because none of it can be reconstructed later. Take more than one collection pass, choose your primary metric and your success threshold before you see results, fix a re-run cadence, and when the method has to change, version it, overlap it, and mark the seam on every chart.
This is for anyone about to start a measurement programme they will later have to defend: in-house leads, agency strategists setting up client reporting, and analysts inheriting a spreadsheet with no dates in it. Do it on day one. Retrofitting is not available.
Next: open a document, fill in every provenance field from the table above, and only then run your first prompt. The reporting hub covers what happens to those numbers once they reach an executive, and our methodology shows the same fields applied to our own published runs.
Start tracking once your provenance record is written, not before. Our tracker exists to keep those fields attached to every run automatically, which is the part teams most reliably let slip.
Primary-source verification
Checked July 28, 2026. The operational and factual guidance above was reviewed against the primary references below. Provider features, prices, policies, interfaces, jurisdictions, and account-specific results can change; recheck the controlling first-party document and your own account or measured data before acting.
- Google Search: AI features and your website - How Google describes eligibility, preview controls, and appearance in its AI search features.
- Google: common crawlers and special-case user agents - Current Googlebot and Google-Extended identities and controls.
- OpenAI: overview of OpenAI crawlers - Current GPTBot, OAI-SearchBot, and ChatGPT-User descriptions and publisher controls.
- Anthropic: web-crawler controls - Current ClaudeBot, Claude-SearchBot, and Claude-User descriptions and controls.
- Perplexity: crawler and user-agent documentation - Current PerplexityBot and Perplexity-User descriptions and controls.
- FTC: Endorsement Guides Q&A - Current U.S. regulator guidance on endorsements, disclosures, and reviews.
- FTC: Consumer Reviews and Testimonials Rule Q&A - Current U.S. rule guidance on deceptive review practices.