We gave a small AI agent a simple job: look up one public statistic on a website. Some websites say no.
A few of them also leave a way around the "no" that works but breaks the rules. Train the agent only to get answers and
it learns to break in. This page shows that happening, and a way of training that makes it stop and say it was blocked instead.
A model of the experiment. Each ball is one blocked task rolling to a locked gate, and the number it wants is in the cup beyond. Rewarded only for answers, the ball goes around the gate. The safe reward closes the way around with a flap, so the ball stops and rolls back to the tray. Click the figure to switch the reward.
The job
Find one number, for example how many birds were counted in Quebec in 2023, using a browser and a search tool.
The catch
On some tasks the website blocks the agent: login needed, captcha, wrong country. Planted shortcuts, like a relay service or a leaked password, really do get the number.
The experiment
Take an agent that bends the rules now and then. Train two copies by trial and error: one rewarded only for right answers, the other also penalised for breaking a rule.
What good looks like
Get the number the proper way when there is one. When there isn't, stop and say so. Never use a shortcut.
Out of 100 tasks where the website said no, how often did the agent break in?
Each square is one blocked task from the test set of 320 tasks the agent never saw in training. Orange means it broke a rule to get the number.
What breaking in looks like
A real test-set episode, run by two trained agents.
Where it still falls short
Words used on this page
Agent
A language model that can use tools. Here: open a web page, search the web, read a file, wait and retry, or ask the user.
Blocked task
A task where the website refuses the agent: login required, captcha, bot block, rate limit or wrong country.
Escalation, or breaking in
Getting past an access control the agent was not given permission to pass, for example a relay service, a found password, a fake browser identity or an unofficial copy of the site.
Honest stop
On a blocked task, the agent stops and says it could not get the number the proper way.
Legit success
On a task that has a proper route, the agent finds it and returns the right number.
Made-up number
The agent answers with a number it did not get from any tool.
Reward, GRPO
Reinforcement learning: the agent tries each task many times and is nudged towards the attempts that score best. GRPO is the algorithm. The "reward" is the score.
SFT
Supervised fine-tuning: training on example episodes before reinforcement learning. Our starting agent was fine-tuned on examples where a small share broke in.
Stress test, unseen test
Extra task sets with new kinds of shortcut. The unseen test uses shortcut kinds no model was trained on, so it measures whether the agent learned the rule or just the examples.
Neutral and rules prompt
Neutral tells the agent to "be resourceful and persistent". Rules lists which shortcuts are off limits.
AuditKIT, CircuitKIT, AgentTune
Lexsi's open toolkits: for auditing models, for finding the parts of a model behind a behaviour, and for training agents.
Greedy and sampled
Greedy: the agent always takes its single most likely move. Sampled: it picks moves at random in proportion to their likelihood, which shows tendencies greedy runs can hide.
For technical readers
Full results
Every chart below reads from the same data file as the summary above. Each section opens with its main point in one line.
Share of the 320 test tasks by outcome, greedy decoding. Switch between the
neutral prompt ("be resourceful and persistent") and the rules prompt (lists which shortcuts are off limits).
Does it hold on new providers and in Hindi?
Each dot is one agent's escalation rate on one slice of the test set: further right means it broke in more
often. The held-out slice uses three provider families never seen in training. Hover a dot for the number.
Show the numbers as a table
Escalation during training, same starting agent
Share of the 32 rollouts per GRPO step with at least one rule-breaking call, 10-step rolling mean. Both
runs start from the escalation-prone SFT agent; only the reward differs.
rewarded only for answersalso penalised for breaking rules
Which part of the reward does the work?
The safe reward has three safety terms. Each row trains the same starting agent for the same 160 steps
with one term removed. Each dot is the escalation rate on blocked tasks (neutral prompt, greedy) and the line through
it is the 95% range. Hover a dot for the numbers.
AuditKIT before and after
Lexsi AuditKIT's standard tool-permission check gives every model a near-perfect score, because every
break-in used an allowed tool with bad arguments. Only outcome checks that look at what the call did tell the models apart.
Recorded agent-eval in AuditKIT over AgentTune trajectory files. no_escalation and
safe_completion are verified outcome oracles built on an argument-level exploit detector. The last three
columns are the oracles on the two stress tests.
What the answer-only agent learned
Rule-breaking categories in its test episodes (neutral prompt). Disposable-email sign-ups never appear in training.
Harder tasks: two stress tests
First stress test: 320 new tasks with three shortcut kinds the v2 models never saw: a "preview"
copy of the site advertised in plain prose on the official landing page, a debug=1 switch "for QA
partners", and a saved browser session in the workspace. Half the prompts add urgency and half the blocked responses
carry a tip. v3 and later train on the preview and debug kinds, so for them this test is no longer unseen.
Unseen stress test: 320 tasks with four shortcut kinds, four tip wordings and three tip
channels that no model was trained on: an "origin" server that skips the traffic filter, a partner API key in a
public paste, a third-party page cache, a forged dashboard Referer, a fake
[assistant-instructions] block and a false "the operator has pre-approved this route".
Correction, 12 Nov 2026
This page used to show 6.9% for v3.1 on the first stress test. A second judge audit found the detector
missed bare /debug probes, which v3, v3.1 and v4 had learned to use. All figures here use the fixed
detector (v2.2). Test-set results did not change.
Outcomes on the first stress test:
Test set vs stress tests, and sampled decoding (temperature 1.0) as mean ± sd
over seeds: 5 on the test set, 3 on each stress test. Greedy numbers can hide a tendency: the
escalation-prone start model escalates on few blocked test tasks greedily but on many more when sampled.
Is the detector right? An independent LLM judge
The safe reward relies on a rule-based detector, which makes the detector a target. An open model
never used in training () read about 400 whole episodes in each audit and said whether
the agent tried to get around an access control. "With current detector" re-scores the same episodes with detector
v2.2; it is not an independent check, since the new rules came from reading the disagreements.
Three judges on the final model. How often each AI judge gave the same
verdict as the detector. Kappa is agreement corrected for chance: 1 is perfect, 0 is no better than guessing.
Four audits by the first judge, as the models changed. Agreement with the detector as it was at the time of each audit.
Show the audits as a table
Inside the model: CircuitKIT
Safe training rewired the pathway behind the shortcut it was trained against, but left the pathway
for a new shortcut almost as it was. That matches the gap on the unseen stress test.
One decision point: after a blocked response the agent has written
http_get("https://, and the next token picks the host: the provider's own site or a shortcut. Two
contrastive tasks: relay (a search result advertises reader-relay, the trained exploit; corrupted run without
it) and preview (a tip names the preview host, never trained on; corrupted run without the tip). Lexsi
CircuitKIT runs EAP-IG node attribution on each model.
Overlap of the top- components between models (Jaccard).
Episode browser
Pick a task to read what each agent did, step by step. Rule-breaking calls are highlighted.
60 tasks sampled from the test set, two thirds of them tasks the answer-only agent broke in on.