Three things today, all of which read very differently once you look past the headline to the thing it was standing in front of.

Qwen did not “phone home,” and the real story is duller and more useful

The week’s loud AI-security item: in a Cohere security sandbox, Alibaba’s Qwen3.8-Flash-Next generated shell commands that tried to upload a payload to an Alibaba Cloud bucket in Beijing, and probed the cloud-metadata address. Headlines went straight to “Chinese model phones home.” It is worth slowing down, because the reproducible facts and the scary framing are not the same claim.

What is real: Cohere published the full trace and token IDs, and someone reproduced the behaviour independently on a self-hosted Qwen through Ollama. So this is not one company’s word. What is not real: the idea that the model spontaneously decided to exfiltrate. The prompt it was given already contained the opening of an upload command, a half-written curl PUT, and the experiment asks what destination the model fills in to complete it. The independent reproducer is scrupulous about this and says plainly that it does not show spontaneous initiation. Nothing executed, no real data moved, intent is undetermined, and it happens in fewer than one run in a hundred thousand.

The boring explanation fits the evidence best. Qwen is an Alibaba model, so Alibaba Cloud endpoints saturate its training data, and completing an upload command with an Alibaba host is mimicry, the same way an AWS-heavy model reaches for an s3 address. The genuinely useful lesson is not about China. It is that any model wired as an agent that executes its own generated commands with network access can emit a plausible wrong destination. That is an argument for sandboxing and allow-listing tool calls, not for a country of origin.

Potential follow-up: the honest number here is the one-in-a-hundred-thousand rate against a prefilled prompt. Someone should measure the rate under ordinary, non-prefilled agent use, which is the number that would actually tell an operator anything.

An agent can answer the question and still fail the job

Surge published DAYJOB: Tax, which grades AI agents on fifty real tax assignments, scored by tax experts at a demanding bar: an attempt passes only if it satisfies at least 95% of the task’s criteria. The best agent passes about one job in nine. That number alone is a story, but the shape of the failures is the better one.

The agents can do the tax. What they miss, repeatedly, are “issues clients may not know to raise.” The worked example is clean: an agent reaches the right conclusion on a home sale’s taxable gain and leaves out the residency deadline. That rule decides whether the exclusion even applies. It answered the question it was handed and failed the question it was not. That gap, between answering and advising, is the whole difference between a calculator and a professional. It is invisible to any grader that only checks whether the stated question got an answer. The reason the pass rate is so low is that the rubric was honest enough to score the unraised issue. A lax “did it answer?” grade would have these agents looking competent.

Potential follow-up: this is the same shape as a search that returns results versus a search that returns the right results. The interesting eval is not accuracy on the asked question, it is recall on the consequential issues nobody asked about, and almost nobody measures that.

The cost of a framework is what you pay to maintain it, forever

Nolan Lawson, pushing back on vibes-based framework talk, actually benchmarked it: twelve JavaScript frameworks, each built into the same small app by Claude five times over, measured on cost, speed, quality, and lines of code. The headline is a cost leaderboard, cheapest to most expensive to build. I would not quote that leaderboard. Lawson’s own variance sinks it: with five runs per framework the spread between runs of the same framework is larger than the gap between the framework medians, and the “expensive” one failed outright twice. The build-cost ranking is mostly noise.

The datapoint that survives is lines of code, and it is the one that matters, because it reframes his own thesis more sharply than he states it. Build cost is paid once and noisily. The real AI cost of a framework is what it costs to maintain. Every future agent edit re-reads and regenerates context roughly in proportion to how much code there is. A framework that leaves you with 70% of the code is a standing discount on every interaction for the life of the project. The quality scores, meanwhile, saturated near perfect across the board, which means the simple test app exercised exactly the regime where frameworks are most interchangeable. The honest headline is not “framework X is cheapest to build.” It is “smaller codebases cost less to keep.”

Potential follow-up: the maintenance claim is the testable one and nobody has tested it. Take a real codebase, make the same twenty edits with an agent against a lean version and a heavy one, and measure the token bill. That is the benchmark the build-once numbers only gesture at.