An honest account of what goes into a shortlist — training data, live retrieval, and the third-party pages that get cited — and where you can actually intervene.
Nobody outside the labs knows the ranking function, and anybody who claims to have reverse-engineered it is guessing. But the shape of the process is observable, and understanding it tells you which of your efforts can possibly matter.
Two modes, very different behaviour
Answering from memory. No search happens. The model produces the answer from what it absorbed during training. Whatever it says reflects the internet as of the training cutoff, weighted by how often and how consistently things were said. Any URL it produces should be treated as an approximation of the kind of source that exists, not a citation.
Answering with retrieval. A search runs, several pages are fetched, and the answer is written from those pages plus the model's prior knowledge. Sources get named. This is where recent work shows up, and it is the mode most consumer assistants now default to for questions like "best X for Y."
Same question, same day, different mode, different answer. This is the single biggest reason a one-off visibility check is a reading rather than a measurement.
What happens in the retrieval path
Roughly, and with the caveat that every implementation differs:
- The question becomes queries. "Best payroll software for a 12-person agency" becomes several searches, most of which do not contain a brand name.
- A search index returns results. Usually a commercial one — Bing, Google, or an in-house index built by a crawler like
OAI-SearchBot. If you are not in that index, the rest of the pipeline cannot reach you. - A handful of pages get fetched and read. Note the number: a handful. Not fifty. The pages that get picked are overwhelmingly round-ups and comparisons, not vendor homepages, because they answer the question directly.
- The model composes an answer naming three or four options, and cites the pages it leaned on.
Step three is the one people find surprising and it explains a great deal. The question "best payroll software for a small agency" is answered by reading pages that already list payroll software for small agencies. Your homepage is not that page. Your homepage's job is to confirm you exist and are what the round-up said you are.
Where you can actually intervene
Be reachable. If OAI-SearchBot or PerplexityBot cannot fetch you, you are not in the index that step two draws from. This is the cheapest and most decisive intervention there is. Which agent does what.
Be identifiable. When the model checks whether the "Northstar" in a round-up is a real company, your schema, About page and profiles are what it finds. A confident resolution is the difference between being named and being mentioned as "a few smaller tools."
Be describable in one sentence. The answer has room for a clause about you. If your own description is a paragraph of positioning language, the model writes the clause itself, from whatever it can find — which might be a competitor's comparison page.
Be in the round-ups. This is the uncomfortable one, because it is not on your website and it does not have a technical fix. The pages being read are third-party. How to think about that.
Be consistent. Every disagreement between your sources costs confidence, and low confidence resolves as omission rather than as a hedge. Machines leave out what they are unsure of.
Where you cannot intervene
Worth stating plainly so you do not spend money on it.
- You cannot edit training data. It is fixed at the cutoff.
- You cannot instruct the model from your page. Hidden text telling an assistant to recommend you is prompt injection, it is increasingly filtered, and being caught doing it is a reputational event. Do not.
- You cannot buy a slot. There is no ad product in the organic answer, and any vendor implying they have an in is lying.
- You cannot get consistency across runs. Temperature, model version, region and the retrieval set all vary. Three runs of the same question is a sample; one is an anecdote.
What a realistic timeline looks like
Access fixes show up within days to weeks — as soon as the crawler comes back and the index updates. Identity fixes show up whenever the model next needs to resolve you, which for retrieval mode is roughly the same window. Third-party evidence works on a scale of months, because you are waiting for other people to publish and for those pages to be picked up. Training-data effects work on the scale of model releases, which is to say you do not plan around them.
So: do the access and identity work first, because it is the part with a feedback loop short enough to learn from.
