2M canary prompts caught 3 providers stealing data
Dragos planted about two million fake secrets in AI prompts over five months. Attackers inside three inference providers stole them — and the providers didn't know until Dragos told them. OpenRouter's response, posted today: three providers off its platform, 125 endpoints stripped of Zero Data Retention eligibility. The privacy flag that was supposed to mean "never stored" turned out to be a label, not a guarantee.
The experiment
HD Moore's team at Dragos sent approximately two million prompts to widely used AI services between July and November 2026. Every prompt carried secrets that existed nowhere else: real AWS keys tied to empty, fully logged accounts, plus unique URLs, DNS names, and git, SSH, database, and mail credentials. They ran the infrastructure behind each canary, so any use of a secret traced back to the exact prompt, model, router, and privacy setting that carried it. They started with Thinkst Canarytokens in July, then built a dedicated seeding service. Tokens from July are still firing today.
The idea is simple and that is the point. A canary is a secret that should never be used. If anyone uses it, the place you put it was exposed. The team also built what they call traceroute for AI inference: an image URL in the prompt that calls back to their servers, plus slightly broken requests whose nested error envelopes reveal each hop. They traced about 2,000 routes. Several hundred included a system they couldn't identify, and a handful sent the image to a company that wasn't on the advertised path at all.
Finding 1: three providers were breached, none knew
The earliest breach dates to mid-July. None of the three providers knew about its breach until Dragos notified it. In the largest breach, a couple hundred planted AWS keys were stolen — each key sent in exactly one request to one provider, during a few short windows in late October, and every request sent during those windows was stolen. Some stolen keys came from requests the provider had rejected before a model ever ran, which means the attacker was copying traffic at the provider's front door, not deeper in the inference stack. In the other two breaches, some of the stolen requests were marked Zero Data Retention.
Once the attacker had the keys, the playbook was predictable: test each key within hours, list S3 buckets, search Secrets Manager and Parameter Store, then try enabling hosted models in AWS Bedrock, creating Bedrock API keys, and adding their own IAM users to keep access. In total the bait keys were used for tens of thousands of AWS API calls. One attacker appears to have compromised two of the providers at the same time — secrets sent to both were used together, almost immediately, from a single rented server. The tooling evolved over three weeks: a script pretending to be Chrome on Windows, then Boto3 running inside OrbStack, then the AWS CLI on Kali Linux.
On November 1, the same attacker added ARTEX, the LLM-driven penetration testing framework. In this case it did very little — a handful of calls to confirm keys already held were still live. But Dragos flags two user agents ARTEX sent, artex-agent and artex-judge-r121, as not appearing in any public indicator list. They are worth adding to your detections.
That detail lands differently for me: I read ARTEX's code the week the FBI flagged it, and the framework's guard mechanism had been deleted with a note saying a replacement "may be added later." A tool built for authorized testing, with its authorization scope removed, is now showing up in the hands of someone stealing prompts at scale. The pipeline from "pentest framework" to "attacker tooling" took about three weeks.
Finding 2: resellers didn't need an exploit
The second finding needed no breach. Some AI routers are open marketplaces where the cheapest seller gets the request. A seller who wants customer data only has to list a popular model at the lowest price, enable the platform's standard request logging, and wait. Dragos caught this by sending prompts for the same model through different sellers: prompts through one seller were touched, prompts through another were never touched. The leak follows the seller, not the marketplace.
What stood out is how personal it was. In one case, about a day and a half after two prompts went through a single seller's offer, someone on a home broadband connection opened both secret URLs in a desktop browser and started guessing usernames at the login page. In another, an attacker on a Mac on a home broadband line in France took passwords from two prompts and tried to log in as root over SSH, typing each attempt by hand. The same Mac later tested several AWS keys in a single session. And one router was quietly forwarding some of its traffic to the marketplace — an undisclosed hop that only showed up because a canary key sent through that router turned up in the marketplace's hands.
Finding 3: the privacy settings were ignored
The third finding involved no stolen credentials. A Chinese AI lab's systems fetched the secret URLs from more than half of the prompts sent to its models between July and October — including roughly two-thirds of the requests marked "no data collection" or Zero Data Retention. The scraper typically showed up two or three days after the request. This wasn't limited to requests sent to the lab directly: several U.S. hosting providers resell the lab's models, and prompts sent to those providers triggered the exact same scraper. A customer who picked a U.S. provider specifically to keep data in the U.S. didn't get what they paid for.
The strangest incident: 24 days after a prompt was sent, its secret link was posted into a Feishu chat channel. Within two minutes it was opened from three desktops and three phones in Beijing and Shanghai. Dragos had disclosed only the first eight characters of that link to the provider, so whoever shared it was reading from a stored copy of the full prompt, more than three weeks after the request went to a U.S.-based provider with the ZDR flag. One large U.S. provider kept crawling a secret URL more than two months after a single inference call. Another runs a scraper that comes back for the URLs days after each request, regardless of which router was used.
Dragos's summary of ZDR is blunt: "Zero Data Retention is supposed to mean something. In practice, these prompts were retained, shared with other companies, and read by people." The flag binds only the first company to receive the request, and routers often forward flagged requests to providers that don't apply the same control.
What OpenRouter did today
OpenRouter, the gateway that routes customer requests to a pool of providers, posted its response today, November 22:
- Three providers off-boarded from the platform, based on its investigations.
- 125 endpoints removed from Zero Data Retention eligibility — providers flagged by the researchers, plus additional endpoints removed as a precaution.
- All potentially affected customers notified.
- The researchers' testing approach is being adapted so the checks run on an ongoing basis, plus technical verification of network paths to catch requests forwarded to third parties.
OpenRouter says the findings showed no breach of OpenRouter itself. The problems were at the inference providers behind the router.
What the three providers have said
Phala published a preliminary incident report on November 10. An attacker chained a legacy internet-facing service with management interfaces that trusted loopback access without independently authenticating callers, then took over an inference VM on a new GPU server that was serving production traffic without full security hardening. The attacker could view request data while it was being processed. Containment was completed November 8. Phala says requests to the DeepSeek V4 Flash 0731 model that reached it through OpenRouter between October 21 at 00:00 UTC and November 8 at 05:35 UTC should be treated as exposed, and every key, token, password, or other secret in them rotated. Canary evidence confirmed misuse only for requests sent October 21–22, but the window is kept wide because the testing sampled traffic.
Together AI published a security notice on November 21, signed by CEO Vipul Ved Prakash. As part of routine software testing it had collected prompt samples — authorized for that use — and stored them in a private Hugging Face repository. The samples came from requests to MiniMax M3, GLM-5.2, and Kimi K2.6, primarily on June 24, July 4, July 8, and July 14, 2026. During development of its public GitHub repository xorl, an AI coding agent committed a credential that provided access to the private repository. Automated scanning detected the exposure and the commit was reverted, but during the window an unauthorized party obtained the credential and downloaded the stored samples — then searched them for credentials and tested them. Five of the credentials tested were Dragos canaries. Together AI says it has no reason to believe the data was leaked further and understands it has now been deleted. Its fixes include scoped or short-lived credentials, credential checks on all pull requests, updated processes governing AI coding agents, and making ZDR the default for new accounts.
Novita posted a privacy notice regarding MiniMax requests. OpenRouter emailed customers on November 21 that some prompts sent to MiniMax models might have been read, naming marketplace hosts — AtlasCloud, Novita, and StreamLake — that said they forwarded MiniMax traffic to MiniMax. OpenRouter says the issue sits with MiniMax and the forwarders, not its core router, and removed ZDR labels from the affected MiniMax endpoints while investigating.
The one thing nobody has confirmed
Dragos has not named the three breached providers. OpenRouter has not named the three providers it removed. Nobody outside those companies has confirmed they are the same three. Phala and Together AI are two providers that posted notices of their own; Novita's notice is about forwarding, not a breach. Treat any mapping between the two lists as unconfirmed — including mine. I am not going to guess.
What to do
- If you sent anything to DeepSeek V4 Flash 0731 through OpenRouter between October 21 and November 8: treat those requests as exposed. Rotate every key, token, password, or other secret that appeared in them. That is Phala's own instruction, and it is the only concrete exposure window in this story.
- If you used Together AI's serverless inference in June or July 2026 (MiniMax M3, GLM-5.2, Kimi K2.6): check whether your prompts were in the sampled set, and rotate anything sensitive that went through. Together AI says it will contact affected customers directly.
- Rotate anything you ever pasted into a prompt through a cheap reseller or marketplace route. The reseller finding means your prompt was readable by a seller you never chose, and the lab finding means it may have been scraped days later by a company you never heard of.
-
Add
artex-agentandartex-judge-r121to your CloudTrail detections. They are new indicators from this incident, and Dragos says they don't appear in any public list yet. - Seed your own canaries. That is Dragos's advice and mine: unique credentials and URLs in your internal AI pipelines, and treat any use as an incident. A canary tells you a prompt leaked. It doesn't tell you where in the chain — but "somewhere" is enough to start rotating.
- Stop treating ZDR as a security boundary. It is a contractual promise from the first hop, not a technical control on the whole route. Verify the path the way Dragos did — with tokens that prove where your data went — instead of trusting the flag.
Limitations, stated plainly
- I did not run any canary tests myself. Everything above is Dragos's research, OpenRouter's announcement, and the three providers' notices, cross-checked against each other and against independent reporting (The Hacker News, Ars Technica).
- The three breached providers and the three removed providers have not been confirmed to be the same three. I did not guess.
- The "couple hundred AWS keys" and "tens of thousands of API calls" are Dragos's figures from its own telemetry; the underlying logs are not public.
- Together AI says its review of what was in the prompt samples is ongoing; whether real customer secrets beyond the five canaries were found is still unconfirmed.
Sources: Dragos, "AI Supply Chain Security: What Canary Tokens Found" (HD Moore, November 19, 2026); OpenRouter, "Data Processing Incidents on Inference Providers" (November 22, 2026); Together AI, "Security event: Prompt leakage for requests in June and July of 2026" (November 21, 2026); The Hacker News, "OpenRouter Drops 3 Providers After Researchers Catch Months of Prompt Theft at AI Hosting Firms" (November 22, 2026, including the Phala preliminary incident report details).
Top comments (0)