Myth: A bigger model fixes bad answers.
Reality: Most bad answers come from retrieving the wrong passages. Fix retrieval first.
Retrieval-augmented generation lets a model answer from your policies, manuals, and contracts, citing the exact passages, and respecting who's allowed to see what.
{
}
Step 01
Split documents by structure, keep titles and metadata, record permissions.
Step 02
Store embeddings and keywords so both meaning and exact terms can be found.
Step 03
Find the most relevant passages the user is allowed to read.
Step 04
The model answers only from those passages and cites them.
Step 05
A test set of real questions scores retrieval and answer quality on every change.
Myth: A bigger model fixes bad answers.
Reality: Most bad answers come from retrieving the wrong passages. Fix retrieval first.
Myth: Vector search alone is enough.
Reality: Part numbers and codes need keyword matching. Hybrid search handles both.
Myth: Just index everything.
Reality: Outdated and duplicate documents produce confident wrong answers. Curate sources.
Every company has answers buried in documents: policies, product manuals, contracts, past tickets, project notes. Traditional search finds documents that contain your words, then leaves you to read them. Large language models can read and summarize, but on their own they don't know your documents and will happily guess.
Retrieval-augmented generation (RAG) combines the two. The system first retrieves the passages most likely to contain the answer, then asks the model to answer using only those passages, with citations. The result is an answer you can check, drawn from your own material, that reflects the latest version of your documents without retraining a model.
The hard part is not the model. It is ingestion that keeps structure and permissions, retrieval that finds the right passage among thousands, and an evaluation set that tells you whether a change made answers better or worse.
Agents and customers get cited answers from manuals, FAQs, and resolved tickets.
Staff ask about leave, expenses, or security policy and see the exact clause.
Find renewal terms, liability caps, and obligations across many agreements.
Engineers query runbooks, specs, and architecture notes in plain language.
Product facts, pricing rules, and case studies at the salesperson's fingertips.
Answers that combine documents with live records from Odoo or your ERP.
| Lever | Why it matters |
|---|---|
| Chunking by structure | Passages that follow headings and sections keep their meaning |
| Hybrid search | Meaning-based and exact-term matching find both concepts and part numbers |
| Re-ranking | A second pass orders candidates by real relevance |
| Metadata filters | Product, region, date, and permission filters narrow the search |
| Answer constraints | The model answers only from sources and says when it can't |
| Evaluation set | Real questions with known answers score every change |
Step 01
List the sources, owners, update frequency, and who may see what.
Step 02
Connect sources, extract text and structure, and store permissions with each passage.
Step 03
Build hybrid retrieval, citation formatting, and refusal when sources don't cover it.
Step 04
Score retrieval and answers on real questions; fix the worst failures first.
Step 05
Release to a pilot group, collect feedback, then widen access.
The biggest pitfall is indexing everything at once. Old policies, duplicates, and drafts produce confident but wrong answers, so we start with curated, owned sources and expand deliberately.
The second is ignoring permissions until late. If access rules aren't captured at ingestion, a knowledge assistant can reveal documents a user should never see, so permissions are part of the pipeline from day one.
The third is measuring quality by gut feel. A few impressive answers in a demo say little; an evaluation set of real questions shows where retrieval fails and whether each change helps.
| Measure | Why it matters |
|---|---|
| Retrieval hit rate | How often the right passage is found for a question |
| Answer faithfulness | Whether answers stay within the cited sources |
| No-answer rate | How often the system correctly says it doesn't know |
| User feedback | Thumbs up or down, with comments, to find gaps |
Picture a support team answering questions about a large product range with manuals, release notes, and years of resolved tickets. Agents search several tools, open long PDFs, and still ask colleagues. A knowledge assistant indexes those sources, respects which documents are internal, and answers with links to the exact page or ticket.
New agents get useful answers on day one, experienced agents spend less time searching, and gaps become visible: questions the assistant cannot answer point to documentation that needs writing.
Success is staff asking the assistant first and trusting what they get back, because every answer shows its sources. Repeated questions to experts drop, documentation gaps become visible, and quality is tracked with the same evaluation set every time something changes.
Retrieval-augmented generation was formalized by Lewis and colleagues as combining a model's built-in knowledge with an external, searchable memory, partly to address provenance and updating knowledgeSource [1]. In practice, that means answers can cite their sources and stay current as documents change.
Quality depends on retrieval. Anthropic reported that adding context to chunks before embedding reduced top-20 retrieval failures by 35%, combining with keyword search reached 49%, and adding reranking reached 67%Source [2]. The same write-up notes that knowledge bases under about 200,000 tokens can simply go in the promptSource [2]. We test the simple option first, then invest in retrieval where it earns its place.
RAG vs fine-tuning
When to use retrieval-augmented generation, when to fine-tune, and when to just write a better prompt. A practical comparison.
AIAI security & governance
Deploy AI safely: private model hosting, data minimization, access controls, prompt-injection defenses, audit logs, and usage policies.
AIConversational BI
Text-to-SQL assistants that answer business questions from your ERP data with transparent queries, read-only access, and verified metrics.
Next step
Thirty minutes with an engineer, no slide deck required. You'll leave with a clearer picture, whether or not we work together.