What Is RAG? Retrieval-Augmented Generation for Business, Explained

RAG, short for retrieval-augmented generation, is a way of making an AI answer from your own documents instead of from its memory. The system first searches your files for the relevant passages, then gives them to the language model along with the question, so the reply is grounded in text you can check. That is the difference between a confident answer and a correct one.

This guide is for an owner or operations lead who has heard the term from a vendor and wants to know what is being sold. It covers how RAG works, the business jobs it suits, how it compares with fine-tuning and a long context window, how permissions must work, what goes wrong and how to test it, and what it costs. We build AI agents for clients, and we start by asking whether you need one at all, so what an AI agent is is a useful companion.

Key takeaways

• RAG searches your documents first and has the model write its answer from what it found, ideally with a link to each source.
• It suits questions that depend on information that changes, such as policies, product documents and contracts. It does not make a model smarter, it makes it better informed.
• The assistant must only retrieve what the person asking is allowed to see, and the check belongs in the search step, with deny by default.
• Most failures are retrieval failures: stale documents, bad splits or the wrong passage. A set of questions with known answers catches them.
• A first agent doing one scoped job starts from £5,000 with us. Running costs depend on your data and usage.

What is retrieval-augmented generation?

Retrieval-augmented generation combines a search step with a language model, so the model answers from retrieved text rather than from what it absorbed in training. The idea was named in a 2020 research paper, Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks by Patrick Lewis and colleagues. The authors noted that language models store facts in their parameters but struggle to access and use that knowledge precisely. Their answer was to pair a pretrained model with a searchable index of documents.

The plain-English version is an open-book exam. A model without retrieval answers from memory, which may be out of date or simply invented. With retrieval, it looks up your handbook first and answers from the page it found. If it cannot find a relevant page, a well-built system says so instead of guessing.

What can a business use it for?

RAG fits any question whose answer sits in documents you own and that people currently search by hand. Three examples are typical, and each is an illustration of the pattern, not a project we have delivered.

• A policy assistant. Staff ask “how many days’ notice for annual leave?” and get the answer from the current HR handbook, with a link to the page.
• A support assistant. A customer asks how to reset a device and the reply comes from the product documentation, with the same link so an agent can verify it. Our AI for customer service guide covers where this works and where a person should step in.
• Contract lookup. A finance lead asks which suppliers have a 60-day payment term, and the assistant returns the clauses, each pointing to its contract.

The common thread is that the answer already exists in writing. If the answer does not exist anywhere in your files, retrieval cannot help, and you are asking the model to guess.

How does RAG work, step by step?

A RAG system has five parts, and each is a place where quality is won or lost.

1. Where the documents live. Usually Microsoft 365, SharePoint or Google Workspace, plus a few systems of record. The assistant reads from these, so what you hold and how tidy it is sets the ceiling.
2. Splitting into passages. Long documents are cut into pieces of a few paragraphs each, so the search can return the part that matters instead of a 90-page PDF.
3. Embeddings. Each passage is turned into a list of numbers that captures its meaning, so “holiday entitlement” and “annual leave” land close together even though the words differ.
4. Search. The question is turned into numbers the same way and matched against the passages. A common choice is PostgreSQL with the pgvector extension, described as open-source vector similarity search for Postgres. It stores vectors alongside your other data and is meant to be used with Postgres full-text search for hybrid search, combining meaning with exact keywords.
5. Answer with citations. The best passages go to the model with the question, and the model writes the reply and points to its sources.

Search quality matters more than most buyers expect. Anthropic’s engineering team describes a method called contextual retrieval, which prepends explanatory context to each passage before embedding. In its tests, combining that with keyword search cut the top-20 retrieval failure rate by 49%, and adding reranking cut it by 67%. Those are one vendor’s test results on its own data, so treat them as evidence that the search step is worth engineering, not as a number you will reproduce.

RAG vs fine-tuning vs a long context window

Choose by what you are trying to change. RAG supplies facts at answer time, fine-tuning changes the model’s behaviour, and a long context window simply fits more text in one request.

Approach	What it changes	Fits when
RAG	The facts the model sees	Documents change often
Fine-tuning	How the model behaves	You need a style or format
Long context	Nothing, text goes in whole	The document set is small

On fine-tuning, open models from places like Hugging Face, which hosts over 2M models, can be adapted to a task with your own examples. That helps when you need consistent labels or a house style. For policies and prices that change monthly, we would reach for retrieval first, because updating a document is easier than retraining a model.

On long context, Anthropic’s engineering team says that if your knowledge base is smaller than 200,000 tokens, about 500 pages of material, you can include it all in the prompt. For a small, stable set, that is simpler than building retrieval. Past that size, or when each question should cost less, or when different people may see different files, retrieval is the better route. We build agents on the Claude API where longer context and steadier judgement earn it, and the model is a per-job decision, as our comparison of ChatGPT vs Claude explains.

How do permissions work in a RAG assistant?

The assistant must only retrieve what the person asking is allowed to see, and the check has to happen in the search step, before any text reaches the model. Our rule is deny by default and opt in to expose.

Microsoft describes the same principle for its own assistant. Its Copilot privacy page says Copilot only surfaces organisational data to which individual users have at least view permissions, and that grounding only accesses content the current user is authorised to access. A custom assistant needs the same discipline.

Three mistakes are common. Indexing everything into one shared pool and trusting the model to “not mention” restricted files fails, because the model has already read them. Copying permissions once at set-up fails, because people change roles. Giving the assistant a super-user account so it can read everything makes every question run with that power. The fix is to carry each person’s identity through to the search and re-check access at query time. If your assistant also takes actions through an MCP server, the same scoping applies there.

What goes wrong with RAG?

Almost all failures are retrieval failures, and the model gets blamed for them. The usual four are easy to name and to test for.

• Stale documents. Last year’s policy outranks this year’s, so the answer is confident and out of date. Version your documents and retire old ones.
• Bad splits. A table is cut in half, or a clause is separated from the definition it depends on. The model gets half a fact.
• The wrong passage. Search returns something similar in wording but not in meaning, and the model dutifully answers from it.
• No source, still confident. The model fills a gap from memory. A good design forces every answer to cite a passage and says “I could not find this” when it cannot.

None of this is solved by a bigger model. It is solved by better documents, better splitting, hybrid search and a refusal path.

How do you test a RAG system?

Build a question set with known answers before you build anything, and run it every time something changes. This is the same discipline we suggest for comparing models in our ChatGPT vs Claude guide.

A workable set has questions people really ask, each with the expected answer and the document it should come from. Include some questions that have no answer in the files, to check the assistant says so. Include some that need two documents. Include some that a given user must not be able to answer, to test permissions. Then check three things separately: did search find the right passage, did the model use it correctly, and did the citation point to it. Marking those apart tells you which part to fix.

Keep the set as a regression test. When someone replaces a handbook or changes the splitting, rerun it. Our agents run inside Python and Django services with logs you can audit. Anything above a risk line, such as sending a message or changing a record, waits for a person to approve it.

What does a RAG assistant cost?

With us, an AI agent doing one scoped job inside your existing tools starts from £5,000, and a rollout of several agents starts from £12,000, or a rolling sprint from £4,000 a month. Those are the published starting prices on our AI agents automation page. What moves the figure is how many sources the assistant reads and how tidy they are, how strict the permissions need to be, and how much sign-off a job needs.

Running costs are separate and depend on your data and usage: the model calls for each question, hosting for the search index, and the time to keep documents current. We prefer to estimate these after seeing your files. For a wider view of pricing, see our guide to what a chatbot costs in the UK.

We do not yet have a client RAG assistant we can show as a case study, so we will not present one. What we do build is agents in Python and Django services, which is the stack our automation work already runs on, and tidy data underneath is a data engineering job before it is an AI job.

A sensible next step

Write down ten questions your team asks that are answered somewhere in your documents, and note where each answer lives. If most live in two or three tidy sources, a first scoped agent is a reasonable bet, and our AI agents automation service starts there. If the answers are scattered and out of date, tidy the documents first, or ask our AI workflow consulting team to map where retrieval would pay off before you spend on a build.