Someone has asked you for a number on AI visibility, and you have opened a blank document to write the questions you will test. That blank document is the single largest source of error in the whole measurement, and it is the part almost nobody documents.
A prompt set is not a keyword list. It is a sample of the questions your buyers ask assistants, and like any sample it can be drawn well or drawn badly. Draw it badly and every metric downstream inherits the flaw: your share of voice, your citation share, your quarter-over-quarter movement. The arithmetic will be flawless and the answer will still be wrong.
A note on our position before the method: RankRift publishes vendor-neutral criteria for how tracking should be done. That is a real conflict, and we state it on every page where it applies. See our disclosure page for how we handle it. Nothing below requires tracking software, or anyone's.
What a prompt set is a sample of
Statistically, your prompt set is a sampling frame. The population is every question a potential buyer might put to an assistant about your category. You cannot enumerate that population, so you draw a sample and hope it represents the whole.
That framing makes the failure modes obvious. A sample can be too small to say anything. It can be drawn from the wrong population, for example your marketing team's vocabulary rather than your buyers'. It can be systematically skewed toward questions you already win. And it can be silently redrawn between runs, which turns a trend line into an artifact of your own editing.
Three properties make a set defensible:
| Property | What it means | How you check it |
|---|---|---|
| Provenance | Every prompt traces to an observed source, not to a brainstorm | Each row cites where the language came from |
| Stage coverage | Prompts span the buying journey, not just the bottom | Count prompts per stage; look for empty cells |
| Neutrality | No prompt is constructed to produce a flattering answer | An outsider reads the set cold and cannot tell which brand commissioned it |
The neutrality test is the one people skip. Run it. Hand the set to someone in another department and ask them to guess which company built it. If they guess correctly on the first try, the set is compromised.
Step 1: harvest real buyer language
Start with language that already exists in your organisation. You are looking for the words buyers use when nobody is coaching them.
Useful sources, roughly in order of value:
- Recorded sales calls and discovery notes. The exact phrasing of "we're looking for something that does X" is the closest thing you have to a natural-language query.
- Support tickets and pre-sales chat. People type to support the way they type to an assistant: sloppily, in full sentences, with the problem stated before the category name.
- Your own search query data. Site search and search-console queries show category vocabulary, though they are compressed into keyword form and need rewriting into sentences.
- Community and forum threads in your category. Read how questions get asked in public, not how vendors answer them.
- Win/loss interviews. These give you comparison language, which is where assistant answers matter most.
Collect raw quotes into a staging list before you edit anything. Two hundred raw fragments is a healthy starting pool for a mid-sized category. Resist the urge to tidy them up as you go, because the untidiness is the signal.
Then convert fragments into prompts by writing them as a person would actually type them to an assistant: a full question, usually one or two sentences, often with a constraint attached. "Best CRM" is a keyword. "What's a good CRM for a 12-person agency that bills hourly?" is a prompt.
Do not write prompts from your positioning document. Positioning is how you want to be described. A prompt set is a record of how strangers describe the problem before they have heard of you.
Step 2: map prompts to buying stages
Once you have candidate prompts, sort them by where the asker is in their thinking. Stage matters because visibility behaves differently at each one, and because a set weighted entirely to one stage answers only one question.
A workable five-stage split:
- Problem framing. The asker names a symptom, not a category. "Our invoices keep getting paid late, what do people do about that?"
- Category discovery. They now know the category exists and want the landscape. "What kinds of tools handle accounts receivable chasing?"
- Shortlist building. They want names. "What are the main options for automated invoice reminders for small businesses?"
- Comparison. Two or more names are in play. "How do teams choose between an add-on to their accounting software and a standalone tool for this?"
- Validation. They have a preferred option and want reassurance or objections. "What are the common complaints about standalone AR tools?"
Set an explicit target distribution before you fill the cells, and write it into the set's documentation. A common shape puts the weight in the middle stages, because that is where a recommendation is genuinely up for grabs, while keeping real coverage at the edges. Whatever you choose, choose it in advance. Deciding the weighting after you see the results is how sets drift toward flattery.
Stage mapping also protects you from a specific misreading: strong performance at stage five and nothing at stage two usually means assistants know you once your name is supplied, but do not surface you unprompted. Those are different problems with different fixes, and a single blended score hides the distinction.
Step 3: build phrasing and regional variants
Assistant answers can vary with phrasing, and they can vary with locale. Neither of those is a defect you can eliminate. It is variance you have to sample across, the same way you would sample across time of day for a server-latency measurement. Model behaviour on this point should be verified against your own runs rather than assumed. For each core prompt, plan a small number of controlled variants:
- Register. A blunt version ("best AR tool") and a conversational version ("we're drowning in chasing invoices, any suggestions?").
- Constraint. With and without a qualifier such as team size, budget model, industry, or integration requirement.
- Locale and language. If you sell into multiple regions, write region-specific versions rather than assuming one English prompt covers all of them. Currency, regulation, and vendor availability all change the honest answer.
- Framing polarity. Some prompts should invite negatives: "what are the drawbacks of X approach?" A set that only asks for the best of something measures enthusiasm, not standing.
Two to four variants per core prompt is usually enough to reveal whether a result is phrasing-sensitive. If a brand appears under one register and vanishes under another, that instability is a finding worth reporting, not noise to be averaged away silently.
There is no universal set. A prompt set built for project management software will not transfer to industrial fasteners or to legal services, because the vocabulary, the stage structure, and the regional constraints all differ. Category-specific construction is the whole job. If you want a starting shape rather than a finished artifact, our prompt-set library organises examples by category, and starter prompt sets by category explains how to adapt one without inheriting its assumptions.
Step 4: decide what you will not test, and write down why
Exclusions are part of the method. An undocumented exclusion looks like an oversight later, and a documented one looks like judgement.
Common exclusions, with the reasoning you should record:
- Purely branded prompts. "Is [your brand] good?" almost always returns something about you, because you are named in the question. It measures nothing about discovery. Here is a concrete rejection: a set we would refuse to run includes "Why do teams choose Acme over the alternatives?" (illustrative brand name, illustrative set). The question presupposes teams choose Acme, and the assistant will usually oblige by supplying reasons. That prompt inflates any mention-based metric while telling you nothing you did not already assert. It goes in the excluded list with the note "presupposes the conclusion."
- Prompts nobody asks. Internal jargon, product code names, and phrasing invented in a positioning workshop.
- Prompts with legal or safety sensitivity. Anything touching medical, financial, or legal advice may trigger different assistant behaviour, and interpreting the result is a specialist job. Exclude, or handle separately with qualified review.
- Competitor-disparaging framings. "Why is [competitor] unreliable?" is a leading question. It also produces output you should not be circulating.
- Anything you would not show the assistant provider. A useful ethical shortcut for the whole set.
Keep a visible "excluded prompts" section in the set document, with one line of reasoning each. When an executive asks why you did not test something, you have the answer already written.
For the longer catalogue of ways sets go wrong after they are built, see prompt-set mistakes to avoid.
Step 5: version it and freeze it before you measure
This is the step that turns a list into an instrument.
Before the first run, freeze the set and give it a version number. A minimal header on the file:
- Set ID and version, for example
ar-tools-uk / v1.0 - Date frozen
- Owner
- Stage distribution as planned and as built
- Variant policy
- Excluded prompts and reasons
- Change log, empty at v1.0
Then hold it. Every subsequent run cites the version. If you add prompts, you have created v1.1 and you say so, and you do not compare a v1.1 number to a v1.0 number as if they were the same measurement. Comparing across versions is possible, but it requires either re-running the old set or restricting the comparison to the prompts common to both.
Set size trades directly against cost. Every prompt multiplies by the number of variants, by the number of models, and by the number of repeat runs you need for a stable reading. A set of 40 core prompts with three variants each, across four models, at five repetitions per prompt, is 2,400 individual calls per cycle. Double the prompts and you double the bill and the labour, in both API spend and human review time. That arithmetic is why set size is a design decision rather than an ambition. Pick the smallest set that covers your stages and your regions, and spend the saved budget on repetitions instead, since repetition is what buys you confidence in a non-deterministic system. How many runs you need works through that trade in detail.
Our full construction and sampling rules, including how versions are recorded, live on our methodology.
In Plain English
A prompt set is the ruler you are about to measure yourself with, so build it before you know the score. Harvest real buyer wording from calls, tickets, and queries. Sort it by buying stage and set the weighting in advance. Add a few phrasing and regional variants so you can see how stable a result is. Write down what you deliberately excluded and why, especially any prompt that names your own brand in a leading way. Then freeze it, version it, and only then run it.
This helps anyone who has to defend a visibility number to a sceptical room: an in-house growth lead, an agency strategist building a client baseline, a founder who wants an honest answer rather than a comfortable one. Do it before your first measurement, and redo it properly whenever your category vocabulary shifts. What you should not do is edit the set after seeing results you dislike.
Next step: draft your v1.0 with the five stages above, run the neutrality test on it, and freeze it. Get the prompt-set template pack to start from a structured file rather than a blank page.
Primary-source verification
Checked July 28, 2026. The operational and factual guidance above was reviewed against the primary references below. Provider features, prices, policies, interfaces, jurisdictions, and account-specific results can change; recheck the controlling first-party document and your own account or measured data before acting.
- Google Search: AI features and your website - How Google describes eligibility, preview controls, and appearance in its AI search features.
- Google: common crawlers and special-case user agents - Current Googlebot and Google-Extended identities and controls.
- OpenAI: overview of OpenAI crawlers - Current GPTBot, OAI-SearchBot, and ChatGPT-User descriptions and publisher controls.
- Anthropic: web-crawler controls - Current ClaudeBot, Claude-SearchBot, and Claude-User descriptions and controls.
- Perplexity: crawler and user-agent documentation - Current PerplexityBot and Perplexity-User descriptions and controls.
- FTC: Endorsement Guides Q&A - Current U.S. regulator guidance on endorsements, disclosures, and reviews.
- FTC: Consumer Reviews and Testimonials Rule Q&A - Current U.S. rule guidance on deceptive review practices.