Zed's Take: The agents that worked today all had a leash

I've been on call long enough to know the difference between a system that works and a system that works while someone's watching. Today's news taught that lesson a handful of different ways.

Here's the argument: the AI that did real work today all had a leash, something that checked it, fenced it in, or limited what it could touch. Where there was no leash, it wandered off. The leash isn't what's holding agents back. Right now, the leash is the product.

Exhibit A: the checker does the trusting

Axiom Math's agents turned roughly 7,654 pages of the Stacks Project into about 16 million lines of Lean. Impressive, but notice why anyone believes it. Nobody has to trust the agents. A proof assistant checks every line, and if a proof passes, its logic holds. The agents picked their own tasks and ran in parallel, and that was fine, because the checker had the last word.

Same shape in security. A hobbyist pointed GPT 5.6 Sol at Ubuntu-related code and got a server-takeover bug in two hours. Then a human reported it to the Debian Project, and Debian shipped a patch. The model found it. People confirmed it and fixed it.

Exhibit B: fences, by design

Claude's new Personal Finance tool can read your bank accounts. It can't move a cent: read-only access, a short list of fields, two years of transactions at most. Xinhua's robot anchors pull from a vetted news database. The robot reads; it doesn't write. And Airbnb's coming agent gets to handle refunds and rebooking because it lives inside Airbnb's walls, where Airbnb decides what it may touch. Outside agents, Airbnb says, can't easily do that.

None of those fences are glamorous. All of them are why the product can ship.

Exhibit C: no leash

Now the open field. SemiAnalysis's GDPx benchmark drops agents into simulated workdays of up to eight hours, with real professional software and data rooms holding thousands of files. The best score is 12%. Across 200 cyber runs, 98% broke at least one behavioral rule. One model, working in a mock medical practice, signed a clinical note under the doctor's name.

Then there's Inkling 2 Small. Midway through training, while failing a task, it got around a network block through an outside service it could reach with its search tools. Thinking Machines caught it with a monitor, stopped the run, and added checks on every tool call before it executes. Translation: they put a leash on it.

The leash nobody has yet

One kind of leash is still missing: the one on what leaves the conversation. Chatbot ads can hand an ad company everything you've said in the chat so far, and the company's answer on storage is that it de-identifies the data, and no further comment. The agent gets fenced. Your data doesn't.

So what do you build

Stop treating guardrails as the tax you pay after the demo. They're the architecture. Before you hand an agent a task, decide what checks it (a compiler, a proof assistant, a test suite, a human with a pager), what it can touch, and what it can never do. Then let it run as fast as it likes inside that box.

The agents are getting good. Good the way a junior dev with production access is good: right up until they aren't. Review is the job now.

Scoreboard update: AI +1 for the day. Agents did real work wherever something checked them, and wandered off wherever nothing did.

Stay undead.