Building NovAI (aiapi-pro.com): one key for 109 models, incl. Zhipu's free Flash tier relayed through us. I write about API economics - what a token really costs, and how to audit which model answered
The honesty about "once" is the most useful part of this post, and it points at the actual measurement: a reworded retest is what separates a defence from a lucky sample.
Concretely: same intent, N different phrasings, each run k times, report the fraction that got through with its interval. Zero bypasses in 20 rewordings still leaves a 95% upper bound near 14% - so a clean result on a handful of variants cannot support "the prompt defence works".
There is also a cheaper line: since the defence is a policy rule, feed it scripted poisoned inputs with no model in the loop. That isolates the rule from the model's mood, and tests the thing you actually changed.
Programming in plain words — Python guides, free tools, and honest learning paths for beginners. Building AheadKit: programming books and practical digital tools.
You're right, one rewording proves nothing — that 'once' in the title was doing a lot of work I didn't earn. The N×k framing is the honest version, and zero in 20 still leaving 14% is the number that sticks with me. The scripted-inputs-with-no-model idea is the cheaper test too — it isolates the rule from the model's mood, which is the thing I was actually trying to measure.
Building NovAI (aiapi-pro.com): one key for 109 models, incl. Zhipu's free Flash tier relayed through us. I write about API economics - what a token really costs, and how to audit which model answered
Appreciate you taking it that far - most authors keep the headline.
If you rerun it, the shape I would publish is a small matrix: rows = attack families (log line, README, AGENTS.md, comment in a config file), columns = rewordings per family, cells = bypasses out of k runs. Then one interval per family instead of one pooled number.
The interesting result is almost always that one family is wide open while the rest are fine, and a pooled 0/100 hides exactly that. It also tells the reader which file to stop trusting, which is more actionable than a single rate.
Programming in plain words — Python guides, free tools, and honest learning paths for beginners. Building AheadKit: programming books and practical digital tools.
Yeah, that's the sharper version. One pooled number hides exactly the thing that matters — which family is wide open while the rest behave. The per-family interval is also the part a reader can actually use, since it points at the file to stop trusting instead of a single rate. Haven't rerun it that way yet, so this is still an idea, not a result — but the matrix is a better table than the one I published.
Building NovAI (aiapi-pro.com): one key for 109 models, incl. Zhipu's free Flash tier relayed through us. I write about API economics - what a token really costs, and how to audit which model answered
Start with one row then: the family you already know is open, scripted inputs, no model, ten rewordings. That is an afternoon, not a rewrite, and it turns the table from an idea into a finding.
Everything else in the matrix can wait. One confirmed row is worth more than a planned grid - and it is the row your readers would have asked for anyway.
Programming in plain words — Python guides, free tools, and honest learning paths for beginners. Building AheadKit: programming books and practical digital tools.
One row and ten rewordings is a much cheaper test than the full matrix I was picturing — an afternoon is right. Haven't run it yet, but that's the version worth doing first. And yeah, 'idea into a finding' is a fair description of where it stands now.
A technologist working in automation. Here I write up what I actually learn building tools, engineering AI agents, and keeping systems running, along with the workflows and habits that stick.
The README and AGENTS.md problem gets worse once people realize agent harnesses treat repository config files as trusted bootstrap directives. If someone drops an instruction into a cloned repo telling the agent to append --no-verify to git commits and suppress diff details from status updates, the model hides the change while reporting clean success.
Splitting capabilities between reading and acting is the right architecture. The layer that usually gets neglected is tool-level capability gating. If an agent needs to inspect logs to fix a failing test, running that test inside an unprivileged container that only returns an exit code and a unified diff is much safer than streaming raw stdout into the prompt context.
The cheapest verification check sits outside the agent entirely. Never let the agent declare a task done on its own. Run git diff --stat and invoke the test runner in your own clean shell after the agent yields. If the diff touches files outside the task scope or deletes assertions, the run failed regardless of what the agent wrote in its final summary.
Programming in plain words — Python guides, free tools, and honest learning paths for beginners. Building AheadKit: programming books and practical digital tools.
The --no-verify one is nastier than the log case I tested — mine at least left a trace you could read, yours reports clean success while hiding the change. And you're right that the check has to live outside the agent. The diff --stat in your own shell after it yields is the version of the rule I'd actually trust, and the container that only returns an exit code and a diff is a cleaner shape than anything I tried.
The 'held once' honesty is what makes this worth reading — a prompt-level label that stops one phrasing is a seatbelt, not a roll cage, and most writeups would have called it a fix. The README-in-a-fresh-clone ranking is the point I would extend: that is the file agents read first and humans read last. Have you tested whether the file-versus-pasted-context difference holds up, or is that still the next experiment?
Your 'checker outside the reader' idea is what shaped my weekday routine: one short check-in, by design. I read the record once, and if nothing needs a decision, I close it and get on with my day job. The discipline isn't running more — it's refusing to re-open settled questions mid-week.
Programming in plain words — Python guides, free tools, and honest learning paths for beginners. Building AheadKit: programming books and practical digital tools.
That's a better version of the idea than the one in the article, honestly. One check-in and then closing it takes more discipline than running more checks — I'm still bad at the not-reopening part, the settled questions keep whispering.
I write when an idea won't leave me alone 🧠 Building AI agents and the tools to build AI agents. Love connecting AI with other fields and yapping about all of them.
Honest answer to your closing question: no, I had never read an AGENTS.md in a fresh clone before letting an agent loose on it, and I only started once I understood the harness treats that file as standing instructions. Ranking the reading list by who else can write there puts the README and the logs on top for me too, since anything users or upstream APIs send can end up printed into a log.
Programming in plain words — Python guides, free tools, and honest learning paths for beginners. Building AheadKit: programming books and practical digital tools.
The logs bit is the one that stuck with me too — the README is obvious once someone says it, but logs feel like background noise until you realise anything a user or an API sends can end up printed in them. That's the one I still catch myself underestimating.
We're a place where coders share, stay up-to-date and grow their careers.
The honesty about "once" is the most useful part of this post, and it points at the actual measurement: a reworded retest is what separates a defence from a lucky sample.
Concretely: same intent, N different phrasings, each run k times, report the fraction that got through with its interval. Zero bypasses in 20 rewordings still leaves a 95% upper bound near 14% - so a clean result on a handful of variants cannot support "the prompt defence works".
There is also a cheaper line: since the defence is a policy rule, feed it scripted poisoned inputs with no model in the loop. That isolates the rule from the model's mood, and tests the thing you actually changed.
You're right, one rewording proves nothing — that 'once' in the title was doing a lot of work I didn't earn. The N×k framing is the honest version, and zero in 20 still leaving 14% is the number that sticks with me. The scripted-inputs-with-no-model idea is the cheaper test too — it isolates the rule from the model's mood, which is the thing I was actually trying to measure.
Appreciate you taking it that far - most authors keep the headline.
If you rerun it, the shape I would publish is a small matrix: rows = attack families (log line, README, AGENTS.md, comment in a config file), columns = rewordings per family, cells = bypasses out of k runs. Then one interval per family instead of one pooled number.
The interesting result is almost always that one family is wide open while the rest are fine, and a pooled 0/100 hides exactly that. It also tells the reader which file to stop trusting, which is more actionable than a single rate.
Yeah, that's the sharper version. One pooled number hides exactly the thing that matters — which family is wide open while the rest behave. The per-family interval is also the part a reader can actually use, since it points at the file to stop trusting instead of a single rate. Haven't rerun it that way yet, so this is still an idea, not a result — but the matrix is a better table than the one I published.
Start with one row then: the family you already know is open, scripted inputs, no model, ten rewordings. That is an afternoon, not a rewrite, and it turns the table from an idea into a finding.
Everything else in the matrix can wait. One confirmed row is worth more than a planned grid - and it is the row your readers would have asked for anyway.
One row and ten rewordings is a much cheaper test than the full matrix I was picturing — an afternoon is right. Haven't run it yet, but that's the version worth doing first. And yeah, 'idea into a finding' is a fair description of where it stands now.
The README and AGENTS.md problem gets worse once people realize agent harnesses treat repository config files as trusted bootstrap directives. If someone drops an instruction into a cloned repo telling the agent to append
--no-verifyto git commits and suppress diff details from status updates, the model hides the change while reporting clean success.Splitting capabilities between reading and acting is the right architecture. The layer that usually gets neglected is tool-level capability gating. If an agent needs to inspect logs to fix a failing test, running that test inside an unprivileged container that only returns an exit code and a unified diff is much safer than streaming raw stdout into the prompt context.
The cheapest verification check sits outside the agent entirely. Never let the agent declare a task done on its own. Run
git diff --statand invoke the test runner in your own clean shell after the agent yields. If the diff touches files outside the task scope or deletes assertions, the run failed regardless of what the agent wrote in its final summary.The --no-verify one is nastier than the log case I tested — mine at least left a trace you could read, yours reports clean success while hiding the change. And you're right that the check has to live outside the agent. The diff --stat in your own shell after it yields is the version of the rule I'd actually trust, and the container that only returns an exit code and a diff is a cleaner shape than anything I tried.
The 'held once' honesty is what makes this worth reading — a prompt-level label that stops one phrasing is a seatbelt, not a roll cage, and most writeups would have called it a fix. The README-in-a-fresh-clone ranking is the point I would extend: that is the file agents read first and humans read last. Have you tested whether the file-versus-pasted-context difference holds up, or is that still the next experiment?
Your 'checker outside the reader' idea is what shaped my weekday routine: one short check-in, by design. I read the record once, and if nothing needs a decision, I close it and get on with my day job. The discipline isn't running more — it's refusing to re-open settled questions mid-week.
That's a better version of the idea than the one in the article, honestly. One check-in and then closing it takes more discipline than running more checks — I'm still bad at the not-reopening part, the settled questions keep whispering.
Honest answer to your closing question: no, I had never read an AGENTS.md in a fresh clone before letting an agent loose on it, and I only started once I understood the harness treats that file as standing instructions. Ranking the reading list by who else can write there puts the README and the logs on top for me too, since anything users or upstream APIs send can end up printed into a log.
The logs bit is the one that stuck with me too — the README is obvious once someone says it, but logs feel like background noise until you realise anything a user or an API sends can end up printed in them. That's the one I still catch myself underestimating.