A RAG chatbot in practice: Answers with verifiable sources
The AI assistant on this website may discuss only my work, services, and systems. That requires more than a long system prompt. Prices change, content exists in two languages, and some questions fall outside the published knowledge. The assistant needs to recognise and communicate those boundaries.
This article is the technical case study for the running system. I cover the general difference between fixed processes and autonomous agents in Agentic workflows: How to build reliable AI processes. Here, the focus is the concrete chat pipeline.
1. Knowledge as small, testable records
The knowledge base consists of Markdown files. Each record covers one bounded fact and has a stable ID, title, category, language, and selected search terms. Prices, contact routes, and service limits do not float inside one large prompt.
A schema checks required fields before the records load. Unknown links between records fail early instead of producing an empty source later. This strictness costs a few minutes during writing and prevents ambiguous answers in production.
I deliberately avoided an unfiltered folder of documents. The smaller the relevant knowledge set, the easier it is to understand why an answer was produced.
2. Retrieval with an explicit match state
For every question, the server searches for relevant records. Search terms, synonyms, titles, and body text carry different weights. The result includes both the selected records and a match state. When no record clears the threshold, the model receives that information explicitly.
This changes the response logic. A knowledge gap is not an invitation to guess. The assistant says that it cannot answer reliably and points to the contact route. The system also records the missing topic without exposing its internal marker in the response.
The knowledge base can then grow from real questions. I add a record when the same relevant gap appears repeatedly. Content follows observed demand instead of a guessed list of possible user questions.
3. Model access behind a fixed interface
The application separates retrieval and prompt construction from the model provider. The selected knowledge and the same behaviour rules can therefore be sent through different providers. A fallback remains a controlled server decision rather than an improvised choice made by the model.
The architectural rule is simple: business policy stays in application code. The model writes the answer, but it does not decide whether a price is valid or a source exists. AI system architecture: Interfaces before models explains this separation in more detail.
4. Citation checking during the stream
The server delivers the answer through Server-Sent Events, a stream of typed events sent to the browser. The model may cite knowledge through a defined marker format. A deterministic gate briefly holds incomplete markers and checks their ID against the records available for that request.
A fabricated source ID is removed before it reaches the interface. Valid sources become resolvable links. This does not solve hallucination as a general problem. It does stop an invented reference from looking like evidence.
That boundary matters to me. A second model acting as a fact checker would still make a probabilistic decision. An ID lookup is ordinary program logic and returns the same result for the same input.
5. Security before convenience
The server limits and validates input before an agent session starts. The model has no tools for reading arbitrary files or executing code. Attempts to request internal instructions or secrets are outside its permitted behaviour.
A prompt rule alone would be too weak. Size limits, session checks, provider selection, and the citation gate therefore live in server code. The prompt explains the role. Code enforces the technical boundaries.
OpenAI treats guardrails as a separate layer in its practical guide to building agents. The NIST Generative AI Profile adds lifecycle testing and risk management. For this assistant, those ideas become concrete test cases: prompt injection, missing knowledge, an invalid source ID, malformed input, and an unavailable provider.
What the system deliberately cannot do
The assistant does not access all of my private project knowledge. It cannot promise availability, invent a new price, or enter into agreements. Information that has not been published remains outside the response.
Retrieval does not guarantee truth either. A correctly retrieved record can still be outdated. Every business-critical fact therefore needs a maintained source and a responsible person. Technology can reveal the path of a statement. It does not remove editorial responsibility.
Why I show this architecture
This website currently uses my own systems as the clearest view into my work instead of presenting invented AI client cases. The chatbot demonstrates how I connect knowledge, privacy, providers, and interface in a running product. Its limits are part of that demonstration.
You can talk to the AI assistant to test the system. For a knowledge assistant or a bounded automation pilot in your organisation, the next step is an AI opportunity audit.
About the author
I am Kirk Kleinau, an AI Strategist and Creative Technologist with an IHK qualification in AI management. I develop the assistant, knowledge structure, and website as one connected system. My foundation is eight years of work across design, concept development, project management, web, and frontend engineering.