Trust as architecture: why AI agents should suggest, never act

Fri Sep 4 2026
Technology
LLM
Topic
Agentic AI
AI Governance
TL;DR: AI agents that act inside real systems need more than usage policies to be trustworthy, because policies only get followed some of the time and reviewed even less. We built an internal platform on three rules baked into the architecture itself: agents can only suggest changes, never make them; anything derived from confidential data is automatically confidential too; and every AI step is metered and capped as it runs, not reported on afterward. The result is a system where the safe path is the only path, so trust does not depend on anyone remembering the rules.

Most organizations meet AI twice. The first meeting is a chat window: you ask a question, it answers, and nothing else happens. The second meeting is more ambitious. AI that does not just answer but acts: software that reads your systems, drafts the report, updates the record, sends the reminder. The industry calls this second kind an AI agent: a program that uses AI to carry out multi-step work with some independence, the way you would delegate a task to a colleague rather than look up a fact.

Agents are where the real value is, and they are also where the question changes. With a chat window the question is "what can it do?" With an agent the question becomes "how do we know what it did?" The first question is exciting. The second one decides whether the technology survives contact with real work, because an agent that acts in your systems is no longer a tool you use. It is a participant you have to be able to account for.

Why the usual answer is not enough

The standard way organizations answer that second question is with process. Guidelines for what the AI may do. Review checklists. An approval policy. A dashboard someone is supposed to watch.

All of these share the same weakness: they sample. A reviewer reads some of the output. A policy covers the situations someone thought of in advance. A dashboard shows what someone chose to put on it. Meanwhile the agent works around the clock and produces more than anyone can read. The gap between what is produced and what is reviewed keeps growing, and that gap is exactly where trust erodes. Usually not through one dramatic failure, but through a slow accumulation of small things nobody saw.

At Xomnia we advise organizations on data and AI: we do not recommend what we have not lived. So over the past months we built an internal platform for ourselves and made it lean on AI heavily. It is the place where our colleagues share and find knowledge, and it is where AI agents help run parts of our own operations. Our own company is the test subject.

One small, real example from that platform will carry us through this whole article. Like every company, we have meetings. Our platform turns a meeting recording into a written transcript, produces a readable summary that colleagues can find later, and lets an agent read along for things that were agreed. Suppose the meeting concluded that a consultant's project is being extended. Somewhere, a planning board still shows her becoming available in August, and someone would normally have to remember to fix that. This one everyday flow touches everything that makes agents hard to trust: an AI that wants to change something real, source material that is confidential, and paid AI work happening out of sight. Keep it in mind. Every principle below is this example, protected in a different way.

Building the platform, we adopted one design stance early, and it shaped everything after it:

Trust cannot be a process layered on top of the system. It has to be a property of the system itself.

A comparison helps. A hotel does not hand the cleaning staff a policy document asking them kindly not to open the safes in the rooms. The cleaning card simply does not open safes. Nobody has to remember the rule, check compliance with the rule, or trust anyone to follow the rule, because the rule is built into the key. Whenever we caught ourselves writing down what an agent must never do, we asked the same question: can we make the forbidden thing impossible instead?

That stance became three principles. None of them requires a technical background, and together they are the reason we became comfortable letting AI do more, not less.

The kicker: for an intermittent analytics workload, the vast majority of that spend bought us nothing. We were paying enterprise compute rates around the clock to serve queries that ran for a fraction of the day.

Principle 1: the AI can suggest. Only people decide.

Back to the meeting. The agent reading the transcript concludes that the planning board is now wrong: it says the consultant is free in August, and the meeting just agreed her project runs to September. What happens next is the first principle.

The agent does not change the planning board. It cannot. In our platform, an agent's only possible output is a suggestion. Concretely, the agent produces a suggestion card that reads like this: "Change the availability date on this planning card from August 15 to September 1. Reason: the project extension agreed in Tuesday's meeting. Based on: the meeting transcript and the current card. This is what the card looks like now, and this is what it would look like after." That card lands with a named person who owns planning, in our case the delivery lead. One click approves or rejects. On approval, the system makes the change and files the receipt: who approved what, when, and on the basis of which information.

So when someone later asks "why does the planning board say September?", the answer is never a shrug. It is a name, a reason, and a record.

The important part is why the agent cannot act on its own. It is not because an instruction tells it not to. Instructions can be ignored, misread, or manipulated. It is because no route from the agent to a real change exists, and we built that in two independent layers, on purpose. The first layer: the agent's toolbox simply does not contain a tool that makes changes. It has tools for reading and one tool for submitting a suggestion, and that is all. The second layer: the platform has a single door through which every real change passes, whoever asks for it, and that door checks who is asking and refuses agents outright. Two layers may sound redundant, and that is exactly the point. If a future programming mistake ever put a change-making tool within the agent's reach, the door would still refuse it. Belt and suspenders. Safety-critical industries have worked this way for decades; software around AI should too.

Figure 1: the only route from an agent to a change in the planning board. The agent can produce a suggestion card and nothing else, and the write door applies the change only after the delivery lead has approved it, filing a receipt as it goes.

There is a second half to this principle: how an agent earns trust in the first place. Every new agent starts in what we call practice mode. It does its work for real, on real meetings and real planning boards, reasoning and producing real suggestion cards, but the cards are not delivered to anyone. They are quietly stored. Meanwhile, people keep doing the work the way they always have. After a while we compare: in the situations where the agent wanted to update the board, what did our people actually do? Only when its judgment holds up in that comparison does someone make a deliberate, recorded decision to let its suggestions start reaching the delivery lead for real.

We should be honest about the cost: this is slower than the impressive demos you may have seen elsewhere. Our own agents are, at the time of writing, still in practice mode. Not one of their suggestions has been delivered yet. The final step is a human deciding to trust the thing, and no architecture can take that step for you.

What surprised us is how liberating the design is. Because an agent structurally cannot act, we stopped rationing what it is allowed to think about. We point agents at problems we would never have dared to automate directly, because the worst possible outcome is a bad suggestion that a person declines with one click.

Principle 2: confidential in, confidential out

Stay with the meeting. The transcript of that conversation is confidential: it names a client, a consultant, and the state of a project. Only a small group should be able to read it. But the transcript does not stay alone. The platform writes a summary of it. An agent quotes it in a suggestion card. Next quarter, some overview document may draw on ten transcripts at once. AI systems constantly make new documents out of old ones, and every new document inherits a question from its ingredients: who is allowed to see this?

A summary of a confidential meeting is itself confidential. That sounds obvious, but in most systems nothing remembers it. A person would have to notice, every time a new document is created, and mark it accordingly. People forget. Not often. But a leak only needs once.

Think of it as a security clearance that travels with the data. If any source document is classified as "confidential," every derivative workno matter how small or summarized, automatically inherits that same classification. This is not a manual task left to the user; it is an automated property of the system. The platform watches, from the inside, what every AI step actually reads while it works. The person who wrote the step does not have to do anything for this and cannot skip it, because the watching is part of the machinery every step runs through. When the step writes its result, the result automatically receives the strictest label of everything that went into it. The summary that read the confidential transcript is born confidential, no matter how harmless the summary looks.

Two details make this trustworthy rather than merely clever. First, the system errs in the safe direction: if anything ever goes wrong with the tracking, the result is treated as potentially sensitive and held back from publication, rather than shown. Second, the same tracking answers the question every organization eventually asks of an AI system: why does it claim this? Because every document remembers what it was made from, every statement can be traced back to its sources with a click, instead of with a week of detective work.

A confession from our own history shows why the automatic version matters. For months, those same meeting transcripts were protected only by convention: a filter someone had once written, which every future colleague had to know about and remember not to break. It worked, but it worked the way an unlocked door works in a trustworthy neighborhood. Moving to labels that travel with the data automatically meant the protection stopped depending on memory. That difference, between a habit and a guarantee, is the entire point of this article.

T-Cloud Public heavily markets its use of standard OpenStack APIs as a major selling point, positioning it as a safeguard against vendor lock-in compared to AWS or Azure. And to be fair, this isn't just marketing gloss, T-Cloud Public's OpenStack fidelity is genuinely deep. Terraform providers, openstack CLI tooling, the standard OpenStack ecosystem works against it largely as-is.

In theory, that meant our infrastructure was vendor-agnostic and portable to any other OpenStack-compatible cloud. In practice, we couldn't actually cash in on that benefit.

Why? Because the European cloud ecosystem is shifting under OpenStack's feet. When we decided to leave T-Cloud Public and migrate to STACKIT (the cloud platform operated by the Schwarz Group), we ran straight into a structural wall: STACKIT has deprecated its OpenStack-compatible API, in favor of their own proprietary IaaS API².

As a result:

  • Our "vendor-agnostic" OpenStack configurations could not be reused.
  • We had to write entirely new, provider-specific configurations for STACKIT to rebuild our infrastructure.

Principle 3: the meter is part of the machine

One more time, the meeting. Turning a recording into a transcript, a transcript into a summary, a summary into suggestion cards: every one of those steps costs a little money, because each is a small purchase from an AI provider. A few cents here, a few cents there. Now multiply by every meeting, every document, every retry, every night the system re-checks whether something changed. This is the part of AI systems that nobody puts in the demo.

It matters because AI systems fail differently than normal software. When ordinary software has a bug that makes it work too hard, it gets slow, and you notice. When an AI system has a bug that makes it do too much paid work, nothing gets slow and nothing errors. Suppose a small mistake made the platform believe every transcript had changed every night, so it faithfully re-summarized all of them, every night. It looks exactly like working software. The problem arrives a month later, as an invoice.

We learned this the expensive way, more than once, and drew a firm conclusion: the meter cannot be a monthly report. It has to be built into the machine. In our platform, every AI step is metered at the moment it runs, and budgets are walls rather than warnings: when a limit is reached, the work stops cleanly and waits, instead of quietly continuing while an alert goes unread. And the system refreshes intelligently. When one transcript is corrected, only the summaries and cards that actually depended on that transcript are redone. Everything else is left alone, because the platform knows what was made from what. It is the same ingredient tracking as in principle 2, now saving money instead of guarding secrets.

For a client, this principle translates into a simple test. Ask what happens when the AI budget runs out on a Tuesday afternoon. If the answer involves someone noticing a dashboard, keep asking.

The common idea: one door

Looking back, the three principles are one move made three times: find the one door all the traffic already passes through, and build the guarantee into the door.

Every change in the platform passes through one door, so that is where "suggest, never act" lives. Every AI step runs through one piece of shared machinery, so that is where the ingredient tracking and the meter live. When a guarantee is built into the door, everything built afterwards inherits it automatically. The tenth thing we build on the platform gets all of these protections on day one, and cannot opt out of them. Compare that with the alternative, where every new project must remember to re-implement the rules, and where the rules live in documents that fade as teams change.

Rules written in binders decay. Rules built into the building do not.

Ultimately, trust in AI agents must be a property of the system architecture rather than a process layered on top. Because you cannot review your way to trustworthiness, organizations should focus on building systems where the safe path is the only path. By embedding these guarantees into the architecture, we ensure that AI operations remain bounded, findable, and fixable in one place instead of a hundred.

Written by 

Gijs Mulder

Data Engineer at Xomnia 

Technology
LLM
Topic
Agentic AI
AI Governance
crossmenuchevron-down