Fine-tuning vs retrieval: how an AI actually learns your data

“Train an AI on your own data” is the phrase everyone uses, including us, because it’s the phrase people search for. It covers two techniques that work in completely different ways, and picking the wrong one is the most expensive misunderstanding in this whole space.

Fine-tuning adjusts a model’s weights by showing it examples. It changes how the model behaves.

Retrieval leaves the model alone and instead finds the relevant passages from your material at the moment a question is asked, then asks the model to answer using them. It changes what the model knows right now.

Almost everybody asking for “an AI trained on my documents” wants retrieval and assumes they want fine-tuning, because fine-tuning is the one that sounds like training.

Fine-tuning takes an existing model and continues its training on examples you supply — typically thousands of input-and-ideal-output pairs. The result is a model that has absorbed a pattern: a house style, a specific output format, a classification scheme, a way of handling a particular kind of request.

What it is genuinely good at:

• Making outputs consistently match a format or a tone.
• Teaching a specialised task the base model handles badly.
• Getting shorter prompts, because the behaviour is baked in.

What it is bad at, and this is the part that surprises people:

• Facts. Fine-tuning teaches patterns, not a reliable lookup table. A model fine-tuned on your handbook will sound exactly like your handbook while getting the notice period wrong.
• Updates. Your policy changed? That’s a new training run. Not an upload — a run, with cost and delay attached.
• Citations. Once the material is dissolved into weights there is no source to point at. Nothing can tell you which page an answer came from, because after training there are no pages.

Retrieval — you’ll see it called RAG, retrieval-augmented generation — keeps your material as material. Your documents are split into passages and indexed by meaning. When a question arrives, the passages most likely to answer it are pulled out and handed to the model along with an instruction: answer from these, and if they don’t cover it, say so.

The consequences are all practical:

• Updates are instant. Re-upload the revised document and the next question is answered from it. There is nothing to retrain.
• Answers are traceable. The passage that produced the answer is a real passage in a real document, so the answer can name the page — or, for video, the timestamp.
• Scope is enforceable. Because the model is answering from a supplied set of passages, “don’t answer from anything else” is a meaningful instruction rather than a hope.
• You need no training data. Your documents are already the input. There are no example pairs to write.

The test is one question: are you trying to change what it knows, or how it acts?

• “It should answer questions about our policies / course / archive / manuals.” Retrieval. Every time.
• “It should always reply in this exact structure, in our register, for this specialised task.” Fine-tuning is a candidate — though try a well-written instruction first, because a paragraph of guidance gets most teams most of the way there for none of the cost.
• “Both.” Large teams do combine them. For everyone else it’s a lot of machinery to avoid writing that paragraph.

Ultranivo is retrieval, and that’s a deliberate choice rather than a simplification: it’s what makes an assistant confined to your own data updatable and citable. Tone and boundaries are handled with a custom prompt, which is the cheap solution to the problem fine-tuning solves expensively.

This isn’t an implementation detail your audience will never see. It shows up in the two moments that decide whether they trust the thing.

The first is when they doubt an answer. With retrieval there is a page number to check. With a fine-tuned model there is nothing to check, and their only options are to believe it or leave — which, on anything that matters, means leave.

The second is when you change something. A refund window moves from 14 days to 30. With retrieval you upload the new policy and the assistant is correct within minutes. With fine-tuning, your assistant confidently quotes the old number until you fund a retraining run — and confidently wrong is considerably worse than nothing.

Neither approach removes the risk of a wrong answer entirely, which is its own subject: why AI assistants make things up.