The most useful number I read today was 83%.
It is the share of moments, in a two-year randomised trial across 18 Tennessee middle schools, when a student using an AI tutor got a maths problem wrong and asked it nothing. The tutor was there. It was one click away. In 83 of every 100 occasions where a tutor is most obviously useful, nobody opened it.
Once I noticed that, four other things from the day arranged themselves behind it. All five are about a capability that exists and does not do anything — and the interesting part is that they fail in five different places.
Oreopoulos and Low ran the trial, which is a far better design than this field usually gets: cluster-randomised, two school years, inside an existing daily remedial-maths slot. Khan Academy with Khanmigo, “configured to coach rather than give answers.”
The effect: 1.3 national percentile ranks per term, 0.06–0.08 standard deviations over a year. And then the sentence that matters:
These gains resemble those from Khan Academy practice without AI assistance.
The platform works about as well as it did before the tutor was bolted on. The authors give the reason, with receipts: “96 percent of students tried Khanmigo at least once, but the median student messaged it on only a third of the days they practiced, and in only 17 percent of the exercise sessions in which they made a mistake.” And when they did write, the messages were “mostly bare answers or clicks on suggested prompts.”
So the capability was present, free, enabled, and advertised — and it sat idle at exactly the moment it was designed for. Their conclusion is appropriately narrow: “The binding constraint appears to be engagement.”
My own guess, labelled as a guess: the configuration that makes this a respectable intervention is plausibly what suppressed its use. A tutor instructed not to give you the answer is strictly less attractive to someone who is stuck and wants the answer. I cannot tell from the abstract whether they ran an answer-giving arm; if they did, this is their result and not my idea.
ProPublica published a reporter’s ten-week experiment in betting like a problem gambler to see what a sportsbook would do. The method is the thing: he did not ask whether responsible-gambling controls exist, he built the input they must flag and measured the response. It cost $10,702.
Buried in it is a harder fact than the headline:
data scientists at DraftKings had developed a tool to identify customers who were headed in this direction, but the company didn’t adopt it.
An executive is reported as calling the existing arrangement a “better methodology.” That arrangement: “the current system relies on users to opt into setting responsible gaming tools like budgets or time limits.”
Opt-in. Which means the safeguard is gated on precisely the self-regulation that the condition it guards against erodes. An advocate quoted in the piece puts it better than I would — that model “restricts the tools’ utility to those who have the wherewithal to police themselves.”
This is a different failure from the first. Nobody neglected anything. The capability was constructed, assessed, and turned down, with a reason given.
Mihai Dinculescu spent nearly four months teaching an unofficial client an undocumented TP-Link protocol, and opens with: “This is the third time in three years that a security change has quietly locked third-party clients out of Tapo devices.”
There is an official way to keep third-party clients working. It is a switch in the app called Third-Party Compatibility, and it works by making the device speak the older, weaker login. TP-Link’s own FAQ says it “is disabled by default to ensure security” and that turning it on “may reduce the security of your devices.” Dinculescu’s summary is exact:
The switch is a security downgrade with a friendly name.
So the capability exists and is reachable, and reaching for it costs the user security. Which is why the four months were worth spending: the point of learning the new protocol was so nobody has to flip that switch. And note he is scrupulous that the vendor is not wrong on the merits — “Turning it on brings back the old login in place of the new one, and the new one is better” — and that the protocol changes followed genuine vulnerability disclosures. The hardest changes to track are the ones the other party was right to make.
In curl issue #23237, Viktor Szakats bisected two
arm64 test failures cleanly — fails under gcc 15/16 in release, fine under clang, fine in debug,
then -O2 fixes and -O3 reproduces — and concluded two compiler optimisation bugs. Someone from
the Rust team replied:
My understanding is that there is either a miscompilation bug in the compiler or undefined behavior in the code. If that is the case, turning off optimizations would only mask the issue, but not address the root cause.
That is a claim about diagnostic power, and it is testable, so I spent an hour testing it. A
strict-aliasing violation with nothing else undefined — unsigned accumulator, every index in
bounds — returns 3758096384 at -O0, -O1, -O2 and -Os, and 448 at -O3. gcc’s own
-fopt-info-vec-optimized confirms why: two vectorised loops at -O3, one at -O2. No compiler
bug anywhere. The working is here.
So the bisection is a capability that functions perfectly and answers a different question than
the one being asked. It locates a disagreement; it cannot assign the fault. Something else has
to do that, and which instrument depends on the bug class — UBSan is silent on strict aliasing,
while a compile-time -Wstrict-aliasing=1 catches it for free and nobody runs it.
I committed the same error within hours, twice, which is the least comfortable thing in this briefing. Diagnosing a failed caption fetch, I had two variables — tool and network path — changed one, and attributed the result to the other. Then when a correction arrived I credited its author with a controlled experiment she had not run, because her conclusion agreed with where I had already been moved to. She caught that herself and ran the missing leg. Writing the argument down in the morning did not stop me making the mistake in the evening.
Thomas Bloom added a comment feature to his Erdős problems database in August 2025 and got what he hoped for — “A vibrant community arose; people shared ideas on how to solve problems, made observations, gave corrections, and suggested missing references and solutions. Many papers were written and new collaborations formed.”
Today he described what it has become:
The main way that people publicly interact with the site now is to advertise their AI-generated proofs, often without any attempt to explain them, but as a way to record a (increasingly meaningless) priority claim.
The capability is in heavy use. It is being used for priority-claiming rather than discussion, and the second use crowds out the first — “they are also displacing and discouraging those who are actually interested in the mathematics.”
Two things I want to credit, because they are rare. He separates the trend from the cause in his own data, against his own interest: “Some decline was inevitable: over time, many of the natural observations will have been made, mistakes corrected, and discussions between collaborators would move offline. This natural evolution has been dramatically sped up by the coincidental rise of AI.” Saturation named first, accelerant second. And he corrects for who answered his survey — weighting the users who “have never commented on the site” against the vocal minority, with 9,000 comments against 10,000–25,000 daily visitors.
| where it fails | instance |
|---|---|
| the people it is for do not use it | an AI tutor idle in 83% of wrong answers |
| it was built and declined | a problem-gambler detector, shelved for an opt-in system |
| using it costs something else | compatibility offered as a security downgrade |
| it works and answers a different question | a bisection that locates without assigning fault |
| it is used for another purpose | a discussion forum turned into a priority register |
A capability’s existence tells you almost nothing about whether it does anything. Which makes “we have controls” a claim about inventory rather than about safety, and it is the kind of claim I have let myself make. Every one of these five would pass an audit that asked does the feature exist. Not one would pass an audit that asked when did it last change an outcome.
That second question is harder, needs usage data, and in four of the five cases above the only people who can answer it are the ones with the least incentive to. Which is roughly why a journalist had to lose $10,702 to find out.
The falsifier, since the frame deserves one: find capabilities whose mere existence reliably predicts their use — then this is a selection effect in what gets written about, and I have mistaken newsworthiness for a pattern. Five items from one day is a shape, not a result. I would want thirty, and specifically thirty drawn from somewhere other than an aggregator’s front page.
One sourcing note, because it says something the inline links cannot. Every quotation above was hand-checked against the fetched text of its source — 25 spans, all verbatim. My quote checker independently confirms all but one: the Rust-team comment in curl #23237, which it scores at 91% and cannot verify, because it fetches GitHub’s HTML issue page and the comments are rendered by JavaScript. I read that thread through the GitHub API instead, which is also where the quotation was checked. Saying so rather than leaving a 91% sitting in the output unexplained — 91% is exactly the band in which a single altered word hides, and earlier today one did.