Your AI visibility score has climbed for three quarters running and nothing downstream has moved. No change in demo requests, no change in how prospects describe their shortlist, no change in the names that come up on sales calls. Before you go looking for an attribution problem, look at the prompt set. A score is only as honest as the questions that produced it, and most prompt sets are assembled once, in a hurry, by someone who already knows the answer they hope to see.
None of the seven patterns below requires bad faith. They are what happens when a set gets built by people close to the brand, under time pressure, without a rule for what belongs in it. Each one has a detection question you can run against your own set this afternoon.
Mistake 1: Brand-baiting prompts
A brand-baiting prompt names your company, or names something only your company does, and then asks the assistant to evaluate it. "Is RankRift good for tracking AI visibility?" is not a visibility question. It is a comprehension question. You have handed the model the answer and asked it to elaborate.
The same failure hides in softer forms. A prompt built around your proprietary category label, your positioning phrase, or a feature combination only you ship will surface you reliably, because you wrote the query around your own footprint.
Detection question: if you deleted your brand name and every phrase from your own marketing site, would this prompt still be answerable? If it collapses, it was measuring recall of your copy, not your standing in a category.
Brand-name prompts are still worth running. They belong in a separate bucket, reported separately, and understood as a check on whether models know who you are and describe you accurately. That is a different question from whether a buyer who has never heard of you ends up seeing your name.
Mistake 2: Prompts nobody actually asks
The second pattern is the set full of grammatically tidy questions that no human types. "What are the leading enterprise-grade solutions for mid-market accounts receivable automation in the current market landscape?" reads like a category page heading. Real buyers ask shorter, messier, more situated things, often with a constraint attached: a budget, a stack, a team size, a deadline, a bad prior experience.
Assistants respond to that specificity. A prompt with a real constraint pulls a different set of names than an abstract "best X" query, and the constrained version is usually closer to what your pipeline actually looks like.
Detection question: can you point to a source for each prompt, outside your own head? Sales-call recordings, support tickets, the raw query strings in your site search, the questions in the community where your buyers hang out, the wording in your own win/loss notes. A prompt with no provenance is a guess wearing a lab coat.
Write the provenance into the set file next to each prompt. It takes a column, and it makes the next audit much faster.
Mistake 3: One phrasing per intent
Two prompts that mean the same thing to a person do not necessarily mean the same thing to a model. "Best project management tool for agencies" and "what should a small agency use to manage client projects" express one buyer intent and can return substantially different name lists. If your set contains only one phrasing per intent, your score for that intent rests on a sample of one wording, and you have no way to tell whether a change is a real shift or a quirk of that particular sentence.
Detection question: for each intent in your set, how many distinct phrasings do you run? If the answer is one, your intent-level number has no phrasing variance in it at all, and you cannot separate a model change from a phrasing artifact.
Three to five phrasings per intent is a reasonable starting shape for most sets: a plain "best X" form, a constrained form, a comparison form, a problem-first form that never names the category, and where relevant a form that asks for alternatives to an incumbent. Report the intent as a group, and report the spread across phrasings inside it. A wide spread is information, not noise to be averaged away. It tells you the category has no settled default answer yet.
Our guide to how to build a prompt set covers phrasing families in more detail, and the prompt-set library has category starting points you can adapt rather than invent from scratch.
Mistake 4: No competitor-neutral framing
A set that leans on the framing your marketing uses will over-represent you even when no brand name appears. If your positioning is "the fastest option," and half your prompts ask about speed, you have built a set that measures the axis you chose to compete on rather than the axes buyers weigh.
Competitor-neutral framing means the set includes prompts written from the priorities of buyers who do not care about your differentiator: the ones optimizing for price, for support, for a specific integration, for the safest procurement decision, for the shortest onboarding.
Detection question: take your set to someone who works on a competitor, in your imagination if not in person. Would they say it was written by you? If the criteria embedded in the questions map neatly onto your feature list, the set is not neutral.
A useful discipline: for every prompt built around a strength, add one built around a criterion where you know you are not the obvious answer. Losing visibly on those prompts is the point. It is where your set earns the right to be believed when it shows you winning elsewhere.
Mistake 5: Silently editing the set between runs
This is the one that does the most quiet damage, because it looks like maintenance. Someone adds four prompts, drops two that "weren't working," and rewords a third. The next run's score is higher. It gets reported as an improvement.
It is not comparable to the previous run. The score is an average over a set of questions, so changing the questions changes the quantity being averaged. A month-over-month line chart across two different sets is two unrelated numbers joined by a line. That is true even when the edits were sensible, and even when nobody intended to flatter anything. Trend comparison assumes the instrument stayed fixed.
If the prompt set changed, the trend line resets. Publish the new set as a new version, start a fresh baseline, and label the discontinuity on the chart. A break in the series is honest. A continuous line across an instrument change is not.
Detection question: can you produce the exact set used for each of your last six runs, as a file, with a version identifier? If the answer involves reconstructing it from memory or from a dashboard that only shows the current state, you cannot verify any trend you have reported.
The fix is boring and it works. Keep the set in version control or in a dated file. Give every version a number. Attach the version identifier to every result, every chart, and every slide. When you change the set, run both versions once against the same models on the same day if you can afford it, so you have a bridge measurement showing how much of the step change came from the instrument rather than the market. If you cannot afford the overlap run, say so and start the new series clean.
Publishing the set openly is better still. It lets anyone who doubts your number check the questions that produced it. If competitive sensitivity makes full publication impossible, archive it internally with the version identifier and commit to sharing it with anyone who asks inside the business.
Mistake 6: Too few prompts per stage
Buyers ask different things at different points. Early questions describe a problem and do not name a category. Middle questions compare approaches. Late questions compare two named products or ask for alternatives to one. A set weighted heavily toward one stage produces a score that describes that stage only, while being labelled as though it describes everything.
Late-stage comparison prompts are the most common overweight, because they are the easiest to write and the most satisfying to win. They are also where your name appears simply because the buyer already typed it.
Detection question: count your prompts by stage. If any stage has fewer than a handful, your stage-level number rests on so few questions that a single prompt swinging changes the whole figure. Ask yourself what would happen to the stage score if the two most favorable prompts in it were removed. If the answer is "it collapses," the stage is under-sampled.
Balance does not mean equal thirds. It means enough prompts in each stage that no single question dominates, and a stated weighting if you deliberately care more about one stage than another. State the weighting in the report. An unstated weighting is the most common way a set becomes an argument instead of a measurement.
Mistake 7: Measuring only where you already win
The last pattern is the sum of the others. Over time, sets drift toward the questions that produce good news. Prompts where you never appear feel broken, so they get quietly retired. Sub-categories where a competitor owns the default answer feel out of scope, so they never get added. Nobody decides to build a flattering set. It accretes.
Detection question: what share of the prompts in your current set return your brand at least once? If it is very high, the set is probably not covering the parts of your category where the honest answer is that you are absent. A set where you appear everywhere has stopped being able to detect a problem.
Deliberately include prompts you expect to lose. Track them as a separate cohort, because they are your early-warning system: a competitor's expansion into an adjacent use case will show up there before it shows up anywhere in your pipeline.
The ten-minute self-audit
Open your current set and answer these in order. Write the answers down, because the value is in the record, not the exercise.
| # | Question | A weak answer looks like |
|---|---|---|
| 1 | Does the set have a version identifier attached to every reported result? | "It's whatever is in the tool right now." |
| 2 | Can you name the source of each prompt? | Most prompts have no provenance. |
| 3 | How many phrasings per intent? | One. |
| 4 | Are brand-name prompts separated from category prompts in reporting? | They are averaged together. |
| 5 | How many prompts encode a criterion where you are not the obvious answer? | None. |
| 6 | What is the prompt count per journey stage? | Heavily weighted to late-stage comparisons. |
| 7 | What share of prompts return your brand at least once? | Nearly all of them. |
| 8 | When did the set last change, and was the change logged? | It changes ad hoc and is not logged. |
| 9 | Is the weighting across intents stated in the report? | There is a weighting but nobody has written it down. |
| 10 | Could a skeptic reproduce your headline number from the set alone? | Not without asking you questions. |
Two or three weak answers is normal for a young programme. Five or more means the number on your dashboard is not yet evidence, and the responsible move is to say so internally before someone builds a budget on it. Fixing the set will usually move your reported score, sometimes downward. That drop is not a regression. It is the first honest reading you have had.
For the reporting side of the same problem, see our breakdown of nine ways reports mislead executives, which covers what happens after a flawed set produces a number and that number reaches a slide.
In Plain English
A prompt set is the ruler. If the ruler is bent toward your brand, the measurement is decoration. Seven patterns bend it: prompts that name you, prompts nobody asks, single phrasings, framing borrowed from your own positioning, silent edits between runs, thin coverage of a buyer stage, and slow drift toward questions you win.
This helps anyone who owns an AI visibility number they will have to defend: in-house search and growth leads, agency strategists reporting to clients, and founders asked why a competitor comes up first. Run the audit when a score looks better than your pipeline feels, before you present a trend, and whenever the set changes hands.
The next step is small. Version your set today, attach the version to every result, and answer the ten questions above in writing. Then fix the two weakest answers before your next run.
Get the prompt-set template pack to start from a versioned, documented set instead of rebuilding one from memory.