GUARDRAILS
What stops an AI agent from doing something you cannot undo?
The short answer
Nothing, unless somebody built the stop. An agent that can read your systems can usually write to them as well, and the difference between useful and dangerous is whether a refusal was designed in before the agent was handed the keys. Configuration is not a stop. A setting that can be changed by whoever is doing the configuring is a preference. What follows is how the systems we build are constrained, and where those constraints are written down.
Three things these systems refuse to do
Move money
Payment sending, recipient creation and transfers between accounts are marked human-only for every role. The check runs before the permissions file is read, so raising a role’s autonomy in configuration does not reach them. The banking integration is read and reconcile only. A test asserts this, which means removing the constraint fails the build rather than failing quietly.
Email a person nobody authorised
Permission to contact a human is stored as a record carrying who granted it, when, and on what evidence. Absence of that record is absence of permission, so a newly created row can never receive automated contact. The check runs on every send rather than once at startup, so revoking permission takes effect on the next attempt with no deploy. Do not contact always wins, ahead of any grant.
Fill in a document it is not sure about
When a template needs a fact nobody has confirmed, rendering raises an error instead of producing the document. It used to substitute a dash. A contract with a plausible looking gap is worse than no contract, because the gap is invisible at the moment of signing.
Four questions worth asking any vendor
Is the limit in configuration or in code? A value someone can change under deadline pressure is not a limit.
Does it fail closed or fail open? If a verification secret is missing, does the system reject the request or process it unverified?
Is every decision recorded, refusals included? Log only the successes and you learn about the boundary when somebody complains it did not hold.
Can an agent hear its own output? Without a loop guard, one agent’s comment becomes another agent’s trigger, and the bill arrives before the mistake does.
Why configuration is the wrong place for a limit
Permissions files get edited by the person under the most pressure, at the worst moment, for a reason that seems good at the time. If the only thing between an agent and a bank account is a value in a settings file, the limit exists at the convenience of whoever is having the hardest week. Moving the check into code, ahead of the settings read, turns it from a preference into a property of the system.
Why failing closed costs you something
Failing closed is not free, and it is dishonest to pretend otherwise. A system that refuses when it is unsure will sometimes refuse when it should have proceeded, and a person has to go and unblock it. That is the trade being made. The alternative is a system that proceeds when it is unsure, and the cost of that one arrives in a client’s inbox.
Why the refusals have to be logged too
A record of what was allowed tells you what happened. A record of what was refused tells you the guard is working, and shows you where it is getting in the way. Systems that log only successes are reconstructable after an incident purely by luck, and luck does not extend to every channel.
What this is worth to a regulated firm
If you hold client funds or client files, the question your compliance officer asks is not whether AI is useful. It is what happens when it goes wrong, and who can demonstrate afterwards what occurred. A system whose limits live in code and whose decisions are all recorded can answer that. One whose limits live in a settings file cannot.
How to check any of this yourself
Ask to see where the limit is written. Not the policy document, the line that enforces it, and the test that fails if somebody deletes it. If a vendor cannot show you that, the limit is an intention. If you want to work through how this applies to your own systems, book a Discovery Hour for $250 and bring your hardest example.
hello@syntaxsphere.com