The README-as-orders example lands because the system prompt says 'never expose secrets' and the agent probably complies — 'probably, most of the time' is not a control. The review question shift is the right one: what can actually happen if the model gets talked into something. Do you think the three-hop trace catches the third failure mode on its own, or does least-privilege on tokens have to be a separate gate?
The three-hop trace should catch it, but I’d still treat least privilege as a separate gate. The trace helps identify where the risk is, while least privilege limits the damage if something slips through. Even if the review misses the injected instruction entirely, a read-only token can't approve or merge a PR. I think you need both: trace the path, then make sure the permissions don't exceed what the task actually needs.
That's the cleanest version of the argument — the read-only token turns a review miss from a breach into a non-event, which is exactly what a separate gate should do. The trace finds the risk, the gate caps the damage, and neither one has to be perfect for the pair to hold. One question: does the trace change the permissions you grant up front, or is it mostly auditing them after the fact?
Ideally both, but I see it primarily as a design-time exercise first. The trace should help determine the minimum permissions the agent needs before deployment, not just document what it already has. Then you keep auditing those permissions as the agent evolves and new tools get added. Otherwise, least privilege tends to slowly turn into "whatever made the last feature work."
That's the right ordering — trace first to set the floor, then audit as tools get added, because otherwise every new feature quietly widens the permissions. The drift you describe is the real failure mode: least privilege decays into 'whatever made the last feature work' unless someone re-checks it on a schedule. Do you re-run the trace per release, or only when a new tool lands?
I'd do both, but with different levels of depth. A new tool should always trigger a review because it can change the agent's reach immediately. For releases, I'd make a lighter check part of the normal process, especially if prompts, workflows or credentials changed. The important thing is not to let permission changes slip through as ordinary feature updates without anyone noticing.
Analyst focused on AI security
Adversarial evaluation of local LLMs, censorship and rejection rate, prompt injection testing, log analysis and defensive automation. Visit: sammideblas.com
The authorization framing is the useful one. Context from PR descriptions and CI logs is untrusted input you cannot sanitize, so tool scope and a deterministic check before the write matter more than the wording of the system prompt.
Exactly. You can't realistically sanitize every piece of content an agent reads, and trying to do so creates a false sense of security. I'd focus on making sure untrusted input can never expand tool permissions and that every consequential action passes through an independent policy check.
Good question. I'd make the tool manifest machine-readable and diff it in CI rather than rely on an author flag. Any new capability — write access, network egress, shell, or secret access — should fail the check until an owner approves it. The author's note explains intent, but the manifest diff makes permission drift visible and auditable.
Exactly. I'd also make the approval explicit and tied to the specific permission change, rather than a general PR approval. That way, adding a new tool or widening an existing scope can't hide inside an otherwise routine change. The machine-readable diff gives you something concrete to enforce in CI and something useful to audit later.
Exactly: the decisive question is not whether the model recognizes hostile text, but whether that text can obtain authority it was never granted. Treating every external instruction as data, constraining tools to narrow capabilities, and requiring a durable receipt for side effects creates a boundary that does not depend on perfect classification.
Agreed. The durable receipt is an important part of this too. You need to be able to reconstruct what was requested, which policy was applied, and who approved the action. Otherwise, even if the authorization boundary works, investigating what happened afterward becomes much harder.
This reframes it well — the model proposing something isn't the same as it being allowed to do it. For anything that touches real work, the check that matters is a person approving the action before it goes out, not just a better prompt.
Exactly. A better prompt can help the model make better suggestions, but it doesn't establish authority. For consequential actions, approval needs to happen outside the model, through a policy check or an explicit human decision. The model can recommend the action, but it shouldn't be the one granting itself permission to carry it out.
I build Kelvra is a : Information flow control for AI agents — declare what an agent may know in a file an auditor can read, and get a signed record of what it actually did. Spec stage.
The README-as-orders example lands because the system prompt says 'never expose secrets' and the agent probably complies — 'probably, most of the time' is not a control. The review question shift is the right one: what can actually happen if the model gets talked into something. Do you think the three-hop trace catches the third failure mode on its own, or does least-privilege on tokens have to be a separate gate?
The three-hop trace should catch it, but I’d still treat least privilege as a separate gate. The trace helps identify where the risk is, while least privilege limits the damage if something slips through. Even if the review misses the injected instruction entirely, a read-only token can't approve or merge a PR. I think you need both: trace the path, then make sure the permissions don't exceed what the task actually needs.
That's the cleanest version of the argument — the read-only token turns a review miss from a breach into a non-event, which is exactly what a separate gate should do. The trace finds the risk, the gate caps the damage, and neither one has to be perfect for the pair to hold. One question: does the trace change the permissions you grant up front, or is it mostly auditing them after the fact?
Ideally both, but I see it primarily as a design-time exercise first. The trace should help determine the minimum permissions the agent needs before deployment, not just document what it already has. Then you keep auditing those permissions as the agent evolves and new tools get added. Otherwise, least privilege tends to slowly turn into "whatever made the last feature work."
That's the right ordering — trace first to set the floor, then audit as tools get added, because otherwise every new feature quietly widens the permissions. The drift you describe is the real failure mode: least privilege decays into 'whatever made the last feature work' unless someone re-checks it on a schedule. Do you re-run the trace per release, or only when a new tool lands?
I'd do both, but with different levels of depth. A new tool should always trigger a review because it can change the agent's reach immediately. For releases, I'd make a lighter check part of the normal process, especially if prompts, workflows or credentials changed. The important thing is not to let permission changes slip through as ordinary feature updates without anyone noticing.
The authorization framing is the useful one. Context from PR descriptions and CI logs is untrusted input you cannot sanitize, so tool scope and a deterministic check before the write matter more than the wording of the system prompt.
Exactly. You can't realistically sanitize every piece of content an agent reads, and trying to do so creates a false sense of security. I'd focus on making sure untrusted input can never expand tool permissions and that every consequential action passes through an independent policy check.
Good question. I'd make the tool manifest machine-readable and diff it in CI rather than rely on an author flag. Any new capability — write access, network egress, shell, or secret access — should fail the check until an owner approves it. The author's note explains intent, but the manifest diff makes permission drift visible and auditable.
Exactly. I'd also make the approval explicit and tied to the specific permission change, rather than a general PR approval. That way, adding a new tool or widening an existing scope can't hide inside an otherwise routine change. The machine-readable diff gives you something concrete to enforce in CI and something useful to audit later.
Exactly: the decisive question is not whether the model recognizes hostile text, but whether that text can obtain authority it was never granted. Treating every external instruction as data, constraining tools to narrow capabilities, and requiring a durable receipt for side effects creates a boundary that does not depend on perfect classification.
Agreed. The durable receipt is an important part of this too. You need to be able to reconstruct what was requested, which policy was applied, and who approved the action. Otherwise, even if the authorization boundary works, investigating what happened afterward becomes much harder.
This reframes it well — the model proposing something isn't the same as it being allowed to do it. For anything that touches real work, the check that matters is a person approving the action before it goes out, not just a better prompt.
Exactly. A better prompt can help the model make better suggestions, but it doesn't establish authority. For consequential actions, approval needs to happen outside the model, through a policy check or an explicit human decision. The model can recommend the action, but it shouldn't be the one granting itself permission to carry it out.
I build Kelvra is a : Information flow control for AI agents — declare what an agent may know in a file an auditor can read, and get a signed record of what it actually did. Spec stage.
Kelvra-depot