Docs/Concepts/Knowledge (RAG)

Knowledge (RAG)

Knowledge gives your agent facts to ground answers on — prices, hours, policies, FAQs, product details. Documents are chunked, embedded, and indexed for semantic retrieval. At every turn the platform retrieves the most relevant chunks and injects them into the system prompt.

What you can upload

Chunking

Documents are split with a structure-aware chunker that follows the document's natural seams:

  1. Split by markdown headings — each heading section becomes a chunk candidate.
  2. Within a section, pack paragraphs up to a configurable target size.
  3. If a paragraph exceeds the ceiling, fall back to sentence boundaries.
  4. If a sentence still doesn't fit, fall back to fixed-width slicing.
  5. Bleed a configurable overlap from each chunk into the next so cross-chunk context isn't lost at a boundary.

Every chunk is stored with its embedding vector, an enclosing-heading reference, and (for PDFs) the page number it came from — enough for the agent to cite its source.

Embedding model

Embeddings are generated with a configurable provider (Gemini by default). The active model is selected via environment configuration, so providers can be swapped without code changes.

At ingestion time, embeddings are batched in parallel to keep large uploads fast — a 200-page PDF typically finishes indexing in well under a minute.

Re-indexing. Changing the embedding model invalidates all existing vectors. Re-upload the document or call POST /v1/documents/:id/reindex per document.

Retrieval — hybrid vector + full-text

Every FAQ turn runs a hybrid retrieval pass that combines two signals:

The two scores are fused with workload-tuned weights, then top-K results above a configurable threshold are returned. Below the threshold, the agent declines to call the LLM at all — preventing hallucinated answers when relevant knowledge isn't present.

If no chunk passes the threshold, the agent does not call the LLM — it returns a deterministic "that's outside what I cover" reply. This prevents hallucinated answers when knowledge is missing.

Source attribution

Retrieved chunks are formatted into the prompt with a structured header so the agent can cite without inventing: 

[Source 1: Pricing Neofir › Plan Studio (p.2)]
<chunk content>

The personality prompt includes a vertical-neutral rule: "When the answer comes from a Context chunk that has a [Source …] header, mention the source in passing. Don't paste the bracket tag itself — read it and attribute naturally." A typical reply: "per the Pricing doc's Plan Studio section, Studio is €45 with 2,000 messages."

usage_mode per document

Each document has a usage_mode setting that controls how the platform uses it:

ValueBehaviourUse for
rag (default) Semantic retrieval per query. Included in prompt only when relevant. Long content (FAQs, policies, manuals).
system Always injected into prompt as "Always-known knowledge", regardless of query. Short, high-priority facts the agent must never miss (hours, contact, refund policy).
policy Injected as guardrail. The agent is instructed it must not contradict. Legal text, terms, disclaimers.
UI for usage_mode picker is not yet exposed. The column exists and the retrieval logic respects it, but the upload form does not yet expose a picker. Until the picker ships, all uploads default to rag mode. You can change the value directly in the database if needed. See Status.

Updating knowledge

Re-uploading a document with the same title deletes the existing chunks first, then re-indexes. The dashboard Documents page shows chunk_count and status per document (pending → indexing → ready or error).

What RAG is bad at

Indexing cost

Embedding cost: $0.00013 / 1,000 tokens with Gemini text-embedding-004 (Mar 2026). A 30-page PDF (≈ 18,000 tokens) costs ~$0.002 to index. Retrieval cost is the embedding of the user query — typically < 100 tokens — and is negligible.