Skip to main content
AI // RAG

Ask your documents a question. Get an answer with receipts.

Retrieval-augmented generation lets a model answer from your policies, manuals, and contracts, citing the exact passages, and respecting who's allowed to see what.

inference.tracelive

{

"summary"
: "Good retrieval beats a bigger model. Cite sources. Respect permissions.",
"sources"
: "Docs · wikis · tickets · ERP",
"search"
: "Hybrid vector + keyword",
"output"
: "Cited answers",
"reviewed_by"
: "a human"

}

How a RAG system answers

  1. Step 01

    Ingest & chunk

    Split documents by structure, keep titles and metadata, record permissions.

  2. Step 02

    Index

    Store embeddings and keywords so both meaning and exact terms can be found.

  3. Step 03

    Retrieve

    Find the most relevant passages the user is allowed to read.

  4. Step 04

    Generate

    The model answers only from those passages and cites them.

  5. Step 05

    Evaluate

    A test set of real questions scores retrieval and answer quality on every change.

RAG misconceptions

Myth: A bigger model fixes bad answers.

Reality: Most bad answers come from retrieving the wrong passages. Fix retrieval first.

Myth: Vector search alone is enough.

Reality: Part numbers and codes need keyword matching. Hybrid search handles both.

Myth: Just index everything.

Reality: Outdated and duplicate documents produce confident wrong answers. Curate sources.

Typical building blocks

Storage

  • PostgreSQL + pgvector
  • Dedicated vector DBs

Models

  • Hosted LLM APIs
  • Open-weight models
  • Embedding models

Pipeline

  • Python
  • FastAPI
  • Background workers

Why search alone isn't enough

Every company has answers buried in documents: policies, product manuals, contracts, past tickets, project notes. Traditional search finds documents that contain your words, then leaves you to read them. Large language models can read and summarize, but on their own they don't know your documents and will happily guess.

Retrieval-augmented generation (RAG) combines the two. The system first retrieves the passages most likely to contain the answer, then asks the model to answer using only those passages, with citations. The result is an answer you can check, drawn from your own material, that reflects the latest version of your documents without retraining a model.

The hard part is not the model. It is ingestion that keeps structure and permissions, retrieval that finds the right passage among thousands, and an evaluation set that tells you whether a change made answers better or worse.

Use cases that work well

Support knowledge

Agents and customers get cited answers from manuals, FAQs, and resolved tickets.

Policy and HR questions

Staff ask about leave, expenses, or security policy and see the exact clause.

Contract lookup

Find renewal terms, liability caps, and obligations across many agreements.

Technical documentation

Engineers query runbooks, specs, and architecture notes in plain language.

Sales enablement

Product facts, pricing rules, and case studies at the salesperson's fingertips.

ERP context

Answers that combine documents with live records from Odoo or your ERP.

What we tune for quality

LeverWhy it matters
Chunking by structurePassages that follow headings and sections keep their meaning
Hybrid searchMeaning-based and exact-term matching find both concepts and part numbers
Re-rankingA second pass orders candidates by real relevance
Metadata filtersProduct, region, date, and permission filters narrow the search
Answer constraintsThe model answers only from sources and says when it can't
Evaluation setReal questions with known answers score every change

From documents to a trusted assistant

  1. Step 01

    Source inventory

    Week 1

    List the sources, owners, update frequency, and who may see what.

  2. Step 02

    Ingestion pipeline

    Week 2–3

    Connect sources, extract text and structure, and store permissions with each passage.

  3. Step 03

    Retrieval and answers

    Week 3–4

    Build hybrid retrieval, citation formatting, and refusal when sources don't cover it.

  4. Step 04

    Evaluation

    Week 4–5

    Score retrieval and answers on real questions; fix the worst failures first.

  5. Step 05

    Rollout

    Week 6+

    Release to a pilot group, collect feedback, then widen access.

Signs your content is RAG-ready

  • Documents have clear owners who keep them current
  • Outdated versions can be identified and excluded
  • Permissions are known for each source
  • There are real questions people ask repeatedly

Common pitfalls we help you avoid

The biggest pitfall is indexing everything at once. Old policies, duplicates, and drafts produce confident but wrong answers, so we start with curated, owned sources and expand deliberately.

The second is ignoring permissions until late. If access rules aren't captured at ingestion, a knowledge assistant can reveal documents a user should never see, so permissions are part of the pipeline from day one.

The third is measuring quality by gut feel. A few impressive answers in a demo say little; an evaluation set of real questions shows where retrieval fails and whether each change helps.

What we measure

MeasureWhy it matters
Retrieval hit rateHow often the right passage is found for a question
Answer faithfulnessWhether answers stay within the cited sources
No-answer rateHow often the system correctly says it doesn't know
User feedbackThumbs up or down, with comments, to find gaps

Who does what

We handle

  • Ingestion, retrieval, and answer pipeline
  • Permissions and source syncing
  • Evaluation, monitoring, and tuning

Your team handles

  • Choosing and owning source content
  • Providing real questions and answers
  • Retiring outdated documents

A worked example

Picture a support team answering questions about a large product range with manuals, release notes, and years of resolved tickets. Agents search several tools, open long PDFs, and still ask colleagues. A knowledge assistant indexes those sources, respects which documents are internal, and answers with links to the exact page or ticket.

New agents get useful answers on day one, experienced agents spend less time searching, and gaps become visible: questions the assistant cannot answer point to documentation that needs writing.

What success looks like

Success is staff asking the assistant first and trusting what they get back, because every answer shows its sources. Repeated questions to experts drop, documentation gaps become visible, and quality is tracked with the same evaluation set every time something changes.

What the evidence says about retrieval quality

Retrieval-augmented generation was formalized by Lewis and colleagues as combining a model's built-in knowledge with an external, searchable memory, partly to address provenance and updating knowledgeSource [1]. In practice, that means answers can cite their sources and stay current as documents change.

Quality depends on retrieval. Anthropic reported that adding context to chunks before embedding reduced top-20 retrieval failures by 35%, combining with keyword search reached 49%, and adding reranking reached 67%Source [2]. The same write-up notes that knowledge bases under about 200,000 tokens can simply go in the promptSource [2]. We test the simple option first, then invest in retrieval where it earns its place.

Frequently asked questions

Will the model make things up?
Grounding answers in retrieved passages and requiring citations reduces this sharply. Evaluation measures how often it still happens.
Can it respect document permissions?
Yes. Permissions are stored with each chunk and applied at retrieval time.
How do updates get indexed?
Incremental sync jobs re-index changed documents on a schedule or via events.
Is RAG better than fine-tuning?
For knowledge that changes, usually yes. Our article on RAG vs fine-tuning covers the trade-offs.
Can it run privately?
Yes, with self-hosted embedding and language models.
Which sources can you connect?
File shares, Google Drive and SharePoint, wikis such as Confluence or Notion, ticketing tools, PDFs, and databases, among others, through their APIs.
How do you measure answer quality?
With an evaluation set of real questions and expected answers, scored for retrieval relevance, faithfulness to sources, and completeness.
What if two documents disagree?
The answer cites both and notes the conflict, and the source report helps owners retire the outdated one.
Can it answer in multiple languages?
Yes. Questions and documents can be in different languages; we test the combinations you need.
How fresh are the answers?
As fresh as the sync schedule: near real time for event-driven sources, or hourly or daily for batch sources.
Where can the assistant be used?
In a web app, inside Slack or Microsoft Teams, in your helpdesk, or embedded in Odoo, depending on where people already ask questions.
Can it show which document an answer came from?
Yes. Every answer cites its sources with links to the exact passage.
What does it cost to run?
Costs depend on the number of documents, questions per day, and model choice. We estimate them in the pilot and monitor them in production.
Can it handle scanned PDFs?
Yes. Scanned documents go through OCR during ingestion so their text can be searched and cited like any other source.
How do you improve answers that miss the right document?
By improving retrieval: contextual chunking, hybrid keyword and vector search, and reranking, measured on your real questionsSource [2].
Can the assistant show where an answer came from?
Yes. Citations to the retrieved passages are a core benefit of RAGSource [1].