AI system architecture: Interfaces before models

The model is the most visible part of an AI system. In production, the architecture decides whether you can isolate a failure, change providers, and trace an answer back to its source. Starting with a model and wrapping the process around its API often ties business rules to a service that changes faster than the business itself.

I start with interfaces. What input arrives? What result does the next step need? Which failure cases must the process distinguish? Only after those contracts are clear do I decide whether a model, a conventional rule, or a combination should do the work.

Treat the model as a replaceable component

A model processes language and makes probabilistic decisions. It should not be the only source of company knowledge or the final authority over every consequence of its output. Separating these responsibilities makes providers replaceable and contains errors.

A small advisory system can keep four concerns independent:

Business rules know the services, approvals, and allowed actions.
Knowledge supplies reviewed content with stable IDs and provenance.
Model access translates a task into the selected provider's format.
Delivery renders text, sources, status, and errors.

This boundary matters more than whichever model currently leads a benchmark. Changing models should not alter product prices, permissions, or URL structure. If it does, too much business logic lives in the prompt.

Anthropic's Building effective agents explains why simple, composable patterns often work better than complicated agent frameworks. That matches my own work. A small router with explicit inputs and outputs is testable. A large agent that controls retrieval, policy, actions, and error handling at once is much harder to inspect.

Define contracts before components

A useful interface defines more than one JSON example. At a minimum, it covers:

allowed fields and size limits
expected response types
known error categories
timeout and retry behaviour
data that may or may not enter logs
the responsible approval point

On this website, a chat request contains a message, bounded history, and locale. Knowledge retrieval returns relevant records plus an explicit match state. The answer reaches the browser as an event stream. Sources, knowledge gaps, and errors remain separate event types, so the interface does not have to infer state from fragments of prose.

That is more work than one direct API call. It saves work as soon as a provider fails or an answer breaks its expected shape. The failure lands at a named boundary instead of moving silently through the whole process.

Manage knowledge outside the prompt

A long system prompt is not a knowledge architecture. Content changes, prices need a validity date, and some facts apply only to one language or service. I therefore manage the assistant's knowledge as small Markdown records with stable IDs, categories, language, and search terms.

Before each answer, retrieval selects only the relevant records. This keeps the context smaller and reveals which source supports a statement. A deterministic citation gate then checks whether every source ID named by the model was actually available. "Deterministic" matters here. Ordinary program logic makes the release decision, not a second model.

The concrete implementation is covered in An agentic workflow in practice. The Systems page documents the underlying projects and their current status.

Give failures names

"Something went wrong" is not enough inside the system. The process should distinguish invalid input, missing knowledge, an unavailable provider, and a security limit. Each category requires a different response.

Missing knowledge calls for a transparent gap. A temporary provider failure may justify a controlled fallback. An invalid permission must stop the process. Sending all three cases back to the same model as another prompt creates a loop, not error handling.

The NIST Generative AI Profile treats risk and evaluation as lifecycle concerns. The architectural consequence is simple: every relevant failure needs an owner, a signal, and a safe response.

Observability begins at the entry point

You can improve a system only when a request remains traceable across its components. OpenTelemetry defines a trace as the path a request takes through a distributed system, with spans representing individual operations. Its documentation on traces also explains how shared context connects those operations.

For an AI process, I would capture at least:

total duration and duration per step
provider and model identifier
retrieval match or knowledge gap
abort reason and error category
retry count
result of each deterministic gate

Prompts, personal data, and secrets should not enter logs without control. Observability means understanding system behaviour. It is not permission to store every input indefinitely.

A monolith is not automatically the problem

I split early prototypes into small services too quickly. That introduced network boundaries, deployments, and failure modes before the domain boundaries were clear. Today I separate responsibilities in code first and move them into distinct processes only when operations or scale require it.

A well-structured modular monolith can be the better architecture for a small team. What matters is that knowledge, model access, rules, and delivery have separate contracts. Process boundaries can come later. Microservices only spread unclear ownership across more machines.

A practical sequence

I use this order for a new AI process:

Record the real user goal and permitted data.
Describe input and output as a schema.
Implement business rules without a model.
Add model access behind a replaceable interface.
Test failures, limits, and human approval.
Only then add routing, more models, or autonomous steps.

The result looks less dramatic than an agent demo. It can be operated, explained, and handed over. That is the standard I use for AI in small and mid-sized companies.

If you want to assess an existing process, begin with the AI opportunity audit. We examine value, data, and architecture boundaries before selecting a model.

About the author

I am Kirk Kleinau, an AI Strategist and Creative Technologist with an IHK qualification in AI management. My own systems separate knowledge, retrieval, providers, and delivery so each part can be tested and replaced. This work builds on eight years of experience across design, concept work, project management, web, and frontend engineering.