Designing a production RAG boundary

Start with the answer contract

A retrieval-augmented generation system is not simply a language model with a search call in front of it. It is a chain of decisions that turns a question into evidence, limits what the model may claim, and leaves enough trace data to explain the result later.

Start by writing the answer contract. For a factual assistant, a useful first contract might be:

• answer only from evidence returned for this request;
• cite the evidence used for each material claim;
• say when the evidence is insufficient;
• keep tenant, region, and access boundaries intact; and
• record the retrieval and generation versions used.

A grounded answer is not one that sounds plausible. It is one whose important claims can be traced to permitted evidence.

This contract changes the architecture. Retrieval quality, authorization, prompt behavior, citations, and refusal are no longer separate enhancements. They are parts of one observable response.

Draw the request boundary

Treat the request as a sequence of typed transitions. Each transition accepts a bounded input and produces evidence for the next decision.

Transition	Input	Evidence to retain
Interpret	Question and request context	Normalized intent and filters
Retrieve	Query, filters, and access scope	Candidate identifiers and scores
Select	Candidates and context budget	Included evidence and exclusion reasons
Generate	Answer contract and selected evidence	Model, prompt, and cited claims
Evaluate	Answer and evidence	Grounding, relevance, safety, and latency signals

The access scope belongs inside retrieval, not after it. Filtering a completed result can reveal that restricted material exists, and it makes evaluation describe a result the user was never allowed to receive.

Make insufficiency a valid result

Production systems need an explicit path for weak evidence. Without it, the generator is rewarded for filling gaps fluently.

type EvidenceDecision =
  | { kind: "sufficient"; passages: Passage[]; traceId: string }
  | { kind: "insufficient"; reason: "no-match" | "low-confidence"; traceId: string };

function decideEvidence(candidates: Passage[]): EvidenceDecision {
  const passages = candidates.filter((candidate) => candidate.score >= 0.78);

  return passages.length > 0
    ? { kind: "sufficient", passages, traceId: crypto.randomUUID() }
    : { kind: "insufficient", reason: "low-confidence", traceId: crypto.randomUUID() };
}

The threshold is not universal. Calibrate it against representative questions, documents, and failure costs. The important design choice is that insufficiency is represented in the interface rather than hidden inside prompt wording.

Evaluate the chain, not only the prose

An end-to-end answer score can tell you that quality changed, but not why. Keep a small evaluation set that lets you separate retrieval from generation.

Measure at least four layers:

1. retrieval relevance: did the candidate set contain the necessary evidence?
2. selection quality: did context construction retain the useful passages?
3. grounded generation: are material claims supported by the supplied evidence?
4. response behavior: did the system cite, refuse, and format as promised?

Run these checks when the document corpus, embedding model, chunking policy, reranker, prompt, or generation model changes. Version those inputs with the result. A score without its system version cannot support a release decision.

Carry evidence into operations

The online trace should answer practical questions without storing unnecessary sensitive text. Prefer stable identifiers, versions, counts, durations, and decision outcomes over complete questions or document bodies.

For each request, record:

• a privacy-safe request or trace identifier;
• corpus, embedding, retrieval, prompt, and model versions;
• applied access and metadata filter identifiers;
• candidate and selected passage counts;
• retrieval, reranking, and generation latency;
• evidence sufficiency and refusal outcomes; and
• bounded evaluation signals sampled for monitoring.

These fields connect an incident to a reproducible system configuration. They also make cost and latency trade-offs visible: increasing candidate depth may improve recall while making reranking slower and generation context more expensive.

Use the release checklist

Before release, verify the boundary as a whole:

• the answer contract is written and testable;
• retrieval enforces access scope before returning candidates;
• insufficient evidence produces a controlled response;
• citations identify the evidence actually supplied to the model;
• evaluation separates retrieval, selection, and generation failures;
• traces contain versions and decisions without unnecessary content; and
• rollback can restore the previous corpus and component versions.

A production RAG system earns trust by making its limits observable. The goal is not to eliminate uncertainty. The goal is to keep uncertainty inside explicit, testable boundaries.