Hit rate, competitor slots and cited sources are the three numbers worth tracking. Here is how to read each one, and how much a single run really tells you.
The first instinct is to type your brand name into ChatGPT and see what comes back. It feels like a measurement. It is not, for the same reason that googling your own name is not a ranking report: you asked a question nobody else asks, and you got an answer shaped by the fact that you named yourself in it.
Here is a measurement that means something.
Ask buyer questions, never brand questions
The unit of measurement is a question a prospective customer would actually type, with no brand name in it.
- ✅ "best product analytics tool for a small B2B SaaS team"
- ✅ "what should I use instead of a full data warehouse for feature usage"
- ❌ "is Northstar Analytics any good"
- ❌ "Northstar vs Amplitude"
The second pair tells you what a model says when you hand it the answer. The first pair tells you whether you are on the shortlist, which is the thing you want to know.
Five questions is a reasonable set: one broad category question, two with a qualifier your ideal customer would use (size, industry, budget), one problem-shaped question that does not name the category at all, and one comparison-shaped question without your name in it.
The three numbers
Hit rate. Of the questions you asked, how many answers named you. That is it. Five questions, named in two, is 40%. Do not weight it, do not build an index, do not turn it into a score out of 100 — the raw fraction is more honest and easier to act on.
Rough bands, in our experience of what they mean:
| Hit rate | Read it as |
|---|---|
| 0 | Not on the shortlist. Usually an access or identity problem, not a marketing one. |
| 20–40% | Known, considered for some framings. Most companies that have done the entity work land here. |
| 40–80% | A default answer in your niche. |
| 80%+ | The category default, or your questions are too narrow. Check the second one first. |
Competitors. Who took the slots you did not. This is the more useful number, because it is a list rather than a metric. The same three names appearing across every answer tells you who the model considers the category, and those are the entity setups worth reading closely.
Sources. Which pages the answers leaned on. Almost always third-party — community threads, review sites, round-ups, directories. This is your content roadmap, handed to you: those are the pages that decide your category, and being absent from them is a specific, addressable gap. What to do about each kind.
How much one run tells you
Less than you want.
Models are stochastic. The same question asked twice can return different lists. Model versions change under you without announcement. Retrieval sets differ by region and by time of day. A browsing run and a memory run answer differently, and you may not be told which you got.
So:
- Do not read a single missing mention as a regression. Ask again.
- Do not compare runs across engines as if they were the same scale. Perplexity's retrieval behaviour is not ChatGPT's; a lower hit rate on one is not a decline.
- Do not chart it weekly. The noise floor is higher than any change your work produced this week, and you will spend the quarter explaining wiggles.
- Do compare quarter to quarter, with the same questions, and treat a change of one question in five as within noise.
What a run is good for is direction and diagnosis. Zero mentions across five questions is a real signal. A competitor appearing in all five is a real signal. The specific sources cited are real information regardless of where you placed.
Reading a zero
A zero hit rate is not usually a content problem, and treating it as one wastes a quarter. Work through it in this order:
- Can the crawler reach you?
robots.txt, WAF, status codes. Fix this first — it is binary and it invalidates everything downstream. - Does the model know who you are? Ask it directly: "what is <brand>?" A hedge or a wrong company means your entity is not resolving, and the fix is identity work, not blog posts.
- Are you in the category conversation at all? Search the same buyer question yourself and read the top round-ups. If your name is not in any of them, that is your answer, and it is a months-long project rather than a week-long one.
Most zeros resolve at step one or two, which is the encouraging part.
What we run
Our AI visibility checker builds a profile of your site, drafts five buyer-style questions from it, puts them to one or more assistants, and reports the hit rate, the competitors and the sources. Free Play gets one engine answering from memory; paid tiers get live-web checks and, on the top tier, every configured assistant at once.
One caveat we build in deliberately: the verdict reads off the best single engine, so fanning out across more assistants can never make the same brand look worse than checking one. And it is a spot check you re-run yourself — there is no scheduled tracking, no stored history of runs, and no alerting. We would rather ship an honest snapshot than a chart of noise.