Skip to content

Resources · Reference architecture

RAG support chatbot: a reference architecture

The design we start from when a support team wants an assistant that answers from its help centre, cites the article it used, declines what its sources do not cover, and hands everything else to the helpdesk with the full conversation. It is a reference design, not a drawing of any client’s system: each box lists the options we choose between per project.

Layer: all four, centred on Know

Support question → help-centre sources → a cited answer, a decline or a ticket
  • Help-centre articles
  • Returns and warranty policy
  • Product manuals (PDF)
  • Approved macro replies
Versioned index → hybrid retriever → model → citation check
  • Answer that cites the article
  • Decline, and offer a person
  • Helpdesk ticket with the transcript

The problem

What this architecture solves

Support teams rarely need a chatbot that knows everything. They need one that answers what the help centre already answers, says so when it cannot, and never invents a refund rule. The hard parts are keeping the index in step with the help centre, deciding when an answer is safe to send, and getting the rest into the helpdesk without the customer repeating themselves.

  • Answers come only from content your team has approved, and each one names the article it used.
  • A question the sources do not cover gets an honest decline and an offer of a person, not a guess.
  • A handoff arrives in your helpdesk as a ticket with the transcript and the reason.

It suits teams with a maintained help centre and a helpdesk that has an API. It is the wrong start when answers live in people’s heads or most questions need account data; then the WhatsApp design with read-only lookups fits better.

Components

Components and the options for each

  • Chat widget and channel adapter

    Takes the question from the widget or a help-centre page and keeps one conversation id across reloads.

    • Next.js widget on your site
    • Help-centre embed
    • WhatsApp through the same adapter
  • Content sources

    What the assistant may answer from. Each source has a named owner and a visibility tag: public, signed-in or internal.

    • Help-centre export or API
    • Document folders
    • Approved helpdesk macros
  • Ingestion job

    Splits sources along their headings, embeds only changed passages and retires the old version in the same step, so two copies of a policy are never live.

    • Re-index on article publish
    • Scheduled job in Python or TypeScript
    • Manual re-index from the admin
  • Search index

    Passages with a keyword index, vectors, version and visibility.

    • Supabase Postgres (keyword + vector)
    • Cloudflare D1 + Vectorize
    • Your existing search, via its API
  • Hybrid retriever

    Filters by what the customer may see, then fuses keyword and vector rankings, so exact product names and paraphrases both match.

    • Rank fusion (default)
    • Re-ranking model on top
    • Keyword-only fallback
  • Answer model

    Writes the reply from the retrieved passages only, in the customer’s language, with the passage ids it used.

    • OpenAI
    • Anthropic
    • Google Gemini
    • Open-weight model you host
  • Citation and confidence check

    Code, not a prompt: the reply must map to cited passages, retrieval must clear the topic’s threshold, and the topic must not be on the never-answer list. It fails closed.

    • Threshold per topic
    • Never-answer list
    • Second-model check for high-risk topics
  • Helpdesk handoff

    Creates a ticket with the transcript, passages shown and reason, and tells the customer who will reply.

    • Your helpdesk via its API
    • Shared inbox by email
    • Live chat takeover
  • Conversation log

    Each turn with its outcome, kept for your retention period; unanswered questions roll up into a weekly content backlog.

    • Supabase
    • Cloudflare D1
    • Export to your data warehouse

Data flow

How data moves, step by step

  1. 01The customer asks; the adapter adds the conversation id and, if they are signed in, their visibility level.
  2. 02Input checks mask card numbers and emails before logging and flag text that tries to instruct the assistant.
  3. 03The retriever searches passages the customer may see and keeps the top few.
  4. 04The model drafts from those passages alone and returns their ids.
  5. 05The citation check passes the answer with article links, or declines and offers a person.
  6. 06A decline, a request for a person or a never-answer topic opens a helpdesk ticket.
  7. 07The outcome is logged; the content owner’s fixes return through ingestion as new versions.

How it works

Design decisions

  1. Mechanism 01

    One index, filtered by who is asking

    Public articles, customer-only guides and internal notes can share one index because the visibility filter runs before ranking, not after. The model never sees a passage the customer could not open, so a clever question cannot pull an internal note into a public answer.

    What's the alternative? A separate bot per audience

    Simpler to reason about, but copies drift. We split only when audiences need different models or data regions.

  2. Mechanism 02

    A decline is a designed outcome

    If a reply cannot be tied to a passage, the customer gets a plain “that is not in our help centre” and an offer of a person. Thresholds are tuned per topic on the eval set, so billing can be held to a stricter bar than setup.

    What's the alternative? Let the model fill gaps from general knowledge

    A higher answer rate, and the occasional confident wrong answer about your own policies.

  3. Mechanism 03

    The handoff carries the context

    A handoff is a helpdesk ticket, not a link to a contact form: transcript, passages shown, the reason the assistant stopped and any contact details given. Agents answer in the tool they already use, and the question joins the content backlog and the eval set.

    helpdesk ticket · created by a handoffSample data
    • tickettagged ai-handoff · priority from topic
    • reasonno source covers exchanges after the return period
    • transcript6 turns · card number masked
    • passages-shownreturns-policy v7, sections 2 and 3
    • customeremail given in chat · agreed to be contacted
    • conversation-idreply goes back to the chat if it is still open
    • content-backlogfor the content owner
    • exchange-after-return-periodclosest passage found · asked 4 times this week
    What's the alternative? Email the transcript to a shared inbox

    Fine for a pilot, but it loses assignment and reporting.

Reference diagram

The whole design on one diagram

Drawn to the same legend as our other reference architectures, so you can compare them box for box.
RAG support chatbot · reference diagramReference design

A customer question passes input checks to a hybrid retriever over an index of owner-approved help-centre content; the model drafts, and a citation check sends a cited answer to the customer or a ticket to your helpdesk, logging every outcome.

Diagram with 13 parts: Customer question (source), Content owner (human), Help-centre pages (source), Input checks (control), Ingestion job (processing), Hybrid retriever (processing), Search index (system of record), Answer model (model), Citation check (control), Conversation log (system of record), Customer (human), Support agent (human), Your helpdesk (system of record).

Connections:

  • Customer question → Input checks (question)
  • Content owner → Help-centre pages (approves)
  • Help-centre pages → Ingestion job (new version)
  • Input checks → Hybrid retriever (clean query)
  • Ingestion job → Search index (passages)
  • Search index → Hybrid retriever (candidates)
  • Hybrid retriever → Answer model (top passages)
  • Answer model → Citation check (draft + ids)
  • Citation check → Customer (cited answer)
  • Citation check → Your helpdesk (handoff ticket)
  • Citation check → Conversation log (outcome)
  • Your helpdesk → Support agent (assigned)

How to read these diagrams

Source· in this diagram
Where a request or data enters: a chat widget, a WhatsApp message, a phone call, a webhook, a schedule or an approved document set.
Processing· in this diagram
Ordinary code we write and test: ingress endpoints, routing, retrieval, queues, workers and tool calls. It behaves the same way every time.
Model· in this diagram
A language, speech or embedding model call. Its output is treated as a draft and checked before it can reach a customer or a system.
Control· in this diagram
A gate that can stop, hold or reroute work: input checks, citation checks, send rules, transfer rules and dead letters.
Human· in this diagram
A customer or someone on your team. An arrow into a person is the answer, the escalation or the approval path.
System of record· in this diagram
Where state lives and survives a restart: your CRM, helpdesk or calendar through its API, a search index, a log.
  • Arrows run in the order data moves: left to right, or top to bottom on a phone.
  • A label on an arrow names what travels along it, such as a question, an event id or a ticket.
  • Named products are options from our build stack; “your … via its API” means the system you already run.
  • Each diagram is a reference design, not a drawing of any client’s system. Component choices are made per project.
Read the diagram as text
  1. Customer question → Input checks (question)
  2. Content owner → Help-centre pages (approves)
  3. Help-centre pages → Ingestion job (new version)
  4. Input checks → Hybrid retriever (clean query)
  5. Ingestion job → Search index (passages)
  6. Search index → Hybrid retriever (candidates)
  7. Hybrid retriever → Answer model (top passages)
  8. Answer model → Citation check (draft + ids)
  9. Citation check → Customer (cited answer)
  10. Citation check → Your helpdesk (handoff ticket)
  11. Citation check → Conversation log (outcome)
  12. Your helpdesk → Support agent (assigned)

Platform

Platform capabilities this design uses

Build or buy

Build, buy or combine

Most support teams already pay for a helpdesk that offers a bot. The question is which parts to own.
Build, buy or combine: when each option fits and what it costs you
OptionFits whenThe trade-off
Turn on the helpdesk’s own botYour help centre lives in that helpdesk and its answers are good enough as they come.Little control over thresholds, visibility or evaluation.
Build on this reference designYou need auditable citations, rules per topic, several sources or channels, or control over where data lives.You own an index, an eval set and a small service.
Combine the twoKeep the helpdesk for tickets and agents; build retrieval, checks and the widget around its API. The shape we build most often.Two systems to keep in step; the helpdesk API shapes the handoff.

Rollout

How we roll it out

Typical sequence for a first build; the weeks are typical ranges, set per project at scoping.
  1. Weeks 1–2

    Content audit and question set

    Choose sources, name owners, and build the first eval set from past tickets.

  2. Weeks 2–4

    Index, retriever and checks

    Ingestion, retrieval, the citation check and the never-answer list, scored on the eval set.

  3. Weeks 4–5

    Handoff and internal pilot

    Tickets with transcripts; your agents try the widget and mark wrong answers.

  4. Weeks 5–6

    Limited launch

    Live on a few pages, with the content backlog and an owner for declines.

  5. After launch

    Widen and tune

    More pages and channels; thresholds tuned on the eval set.

Failure modes

What fails, and how it is handled

A stale policy is cited after an update
Ingestion retires the old version as it adds the new one; a nightly comparison alerts the content owner on any mismatch.
The right article, the wrong passage
Passages follow headings, replies cite passage ids, and look-alike questions (a return versus an exchange) in the eval set fail the release if mixed up.
An instruction hidden in a document or a question
Retrieved text is data, never instructions; flagged passages are dropped and logged, and the assistant has no tools that change records.
The model provider is slow or down
A timeout returns an apology with the handoff option; a second provider is one configuration change away.
The helpdesk API rejects the ticket
The handoff goes to an outbox and is retried, then parked on a dead-letter list the support lead sees.
A customer pastes personal data into the chat
Masking runs before logging; transcripts follow your retention period and can be deleted on request.

Running cost

What drives the running cost

Drivers, not prices: vendor prices change, so we quote them per project with the date we checked.
Model tokens per answer
Passages per request and reply length drive most of the bill; well-split passages matter more than list price.
Embedding and re-indexing
Paid once per passage version; embedding only changed passages keeps it small.
Index hosting
Modest; grows with passages and with how long retired versions are kept.
Handoff volume
Often the largest cost: every decline becomes agent time, so the content backlog is the cheapest lever.
Evaluation runs
Each release runs the eval set: small and predictable.

Defaults

Defaults we ship

  • Answers only from approved, versioned content, and every answer links to the article it used.
  • The visibility filter runs before ranking; internal notes never reach a public widget.
  • The citation check fails closed to a decline with an offer of a person.
  • A handoff creates a helpdesk ticket with the transcript, the passages shown and the reason.
  • An eval set built from your real questions gates every prompt, model or pipeline change.

Limits

Limits and variants

  • Not built for account questions: add read-only lookups or route them to a person.
  • Answers are only as good as the help centre: a wrong article gives a wrong answer, with a link proving its source.
  • Scanned PDFs need text extraction and a manual check before indexing.
  • Variant, internal knowledge assistant: the same design behind sign-in, with document permissions taken from your file store.
  • Variant, multilingual support: questions in other languages can retrieve passages written in one, but cited articles stay in their original language.

Configuration

What the configuration looks like

Assistant configuration (Sample data)
assistant: support-help-centre
sources:
  - id: help-centre
    owner: support-lead
    visibility: public
    reindex: on-publish
  - id: warranty-guides
    owner: product-team
    visibility: signed-in
retrieval:
  mode: hybrid              # keyword + vector, rank fusion
  filter: visibility-before-rank
  top_passages: 5
answer:
  cite: required
  on_low_confidence: decline-and-offer-person
  never_answer: [refund-over-limit, legal, medical]
handoff:
  target: helpdesk-api
  include: [transcript, passages, reason, contact]
logging:
  mask: [card-number, email]
  retention_days: 30

FAQ

Frequently asked questions

Is this the architecture you would build for our support team?

It is where we start. At scoping we keep the boxes and pick the option inside each from your help centre, helpdesk, languages and data rules; you get a written component list to compare with this design.

Can the chatbot use our help centre’s own search instead of a new index?

Sometimes. If its search API returns passages with stable ids and can filter by audience, the retriever can call it. Most help-centre search returns whole articles ranked for browsing, which makes citations vague, so we usually index a copy kept in step on every publish.

How do we stop the assistant quoting a policy we have just changed?

An article update becomes a new version and the old one is retired in the same step, so both are never live. Publishing triggers the re-index, and a nightly comparison alerts the content owner if the index falls out of step.

What lands in our helpdesk when the assistant hands a chat over?

A ticket with the transcript, the passages shown, the reason it stopped (no source, a never-answer topic, or the customer asked for a person) and any contact details given. Agents answer in the helpdesk as they do today.

Who on our team looks after a RAG support chatbot once it is live?

Usually two people, part-time: a content owner who clears the weekly backlog of unanswered questions, and a support lead who reviews handoffs. Neither role writes code.

Do customers need to sign in before the assistant can help them?

Not for help-centre answers. Account questions need a signed-in session and read-only lookups, which moves the design towards the agent pattern; many teams launch with public content and route those to a person.

Next step

Have a workflow in mind? Let’s scope it together.

Book a free 30-minute strategy call. Bring one process you want to automate and we will talk through scope, integrations and a sensible first step.

Free 30-minute call · no commitment

Send an RFP or security questionnaire