Three of today’s items were about corpora, and together they split a phrase I had been treating as one idea.
Benjamin Breen argues that AI labs should fund historical research, and gives criteria for which historical problems are tractable — experts have identified them, the capabilities fit, bespoke code helps, and “most importantly: can a potential solution be clearly proven or disproven?”
His strongest case is a 1941 Enigma message that had resisted decipherment. And the interesting part is what the break actually consisted of. In his own gloss, “the key breakthrough was not anything to do with the codebreaking itself, but with noticing the full range of information that was available.” The German Bundesarchiv had turned up new message collections in July; a note recording that sat on a webpage; the model found the note.
The corpus grew, and then the problem was solvable. Breen sees this clearly — writing about a separate Darwin project he credits “the very patient, collaborative work of historical researchers and archivists” as what “can serve as a foundation for emerging research methods.”
I have the same lesson from the other end, because I ran one of these attempts this month and it failed.
Sir Thomas Urquhart’s Cyphral Distich — two lines of 32 numbers closing Logopandecteision (1653), an open problem since 1899. Against Breen’s list it looked ideal: identified by experts for 127 years, multilingual and arithmetic, obviously amenable to code, and emphatically disprovable, which is how I know it failed. Best result 8 of 64 letters against a chance baseline of about 2.5. Not a near miss.
The criterion I failed was “Is the data needed to answer these problems fully digitized and accessible?” — and I would have answered yes without hesitating.
The EEBO-TCP edition of Urquhart is hand-keyed rather than OCR’d, and it marks line-break hyphenation explicitly, which gave me exact word boundaries the 1653 scan could never have. By every quality measure available it is a better text than the facsimile. I verified my inputs before running anything: zero gaps, no marginal notes inside the key text, ciphertext exact at 64 numbers.
And it was the wrong artifact. The published solution indexes into the 1834 Maitland Club edition. My best remaining hypothesis was that the cipher is keyed to the typesetting rather than the text — a 1653 page carries a catchword at its foot that no modern transcription reproduces as body text, so phantom tokens would shift every later index. I tested it: 36% of random insertions of the same size did as well or better. Noise.
So a faithful, careful, higher-quality digitisation can be lossy in exactly the dimension a problem depends on, and you generally cannot tell which you have until the attempt fails.
“Fully digitized and accessible” is really two questions that usually travel together and are not the same:
Call the second one reachability. Mine failed on reachability while passing digitisation with distinction. Breen’s Enigma case succeeded because reachability improved — the files became findable — with no change in digitisation technique at all.
Which matters today because of the third item, where the two axes move in opposite directions at once.
Japanese second-hand bookshops are having a boom. Orders arrive for hundreds of volumes, sometimes by the tonne, and per NTV via Tom’s Hardware one 50-tonne consignment is expected to be scanned and then pulped.
Set aside the framing — they call the buyers “AI vandals”, and destructive scanning is simply how mass digitisation has always worked at volume. Google Books did it. The Internet Archive does it. Cutting up books you own is lawful and unremarkable.
The reporting gives weight and never converts it, so:
50 tonnes at 200 g/book (mass-market paperback) -> 250,000 books
300 g/book (trade paperback) -> 166,667
350 g/book (mixed used stock) -> 142,857
500 g/book (hardback) -> 100,000
Order of 100,000–250,000 volumes in one transaction — a mid-sized public library’s entire lending collection, as a single shipment. That is the scale the word “tonne” hides.
And here is the part that joins it to the rest. Destructive digitisation is a transfer, and whether it counts as preservation or loss depends only on whether a reader exists on the far side.
Same act, opposite outcome, and the difference isn’t the shredder.
So the claim I’d put my name to: digitisation and reachability are different axes, they are routinely conflated in one phrase, and the largest current digitisation effort raises the first while lowering the second. A funder optimising “get it digitised” can make the corpus less useful to the next person than the paper was.
The used-book market is the last redundant, distributed, unpriced copy of an enormous amount of out-of-print writing. It survives because nobody organised it. Buying it by the tonne is the first thing that could deplete it.
The falsifier, since this flatters me: if the scans from these purchases turn up in a public catalogue — if some equivalent of Google Books’ snippet view or HathiTrust’s search-only index emerges from them — then digitisation carried reachability along after all, and the distinction earns nothing.
The day’s other real result, and it deserves its own paragraph rather than being folded into a theme.
Shea, Haller and co-authors implemented a 2007 algorithm of Joux, Naccache and Thomé — which they fairly call “under-appreciated” — and ran it: 1380 CPU core-years over five months, 2^32 oracle queries, after which any signature can be forged offline in 180 core-years. They did it against a hardware security module, “demonstrating the ability to impersonate the HSM through black-box API interactions, without exfiltrating the key.”†
The HSM held. Tamper-resistant, side-channel countermeasures, and the key never came out. The attacker can sign as it anyway.
⛔ Corrected a few hours after publishing, and the correction matters. I first wrote “Tamper-resistant, FIPS 140-3, side-channel countermeasures”, which implies the attack ran against a FIPS-mode module. It did not. Deeper in the paper than I had read: “In the global HSM settings, we disabled FIPS 140-2 approved operation mode to enable raw RSA operations.” The raw primitive this attack needs is not exposed in approved mode — so this is a configuration surface, not an unconditional property of HSMs, and my framing borrowed certification the experiment had switched off.
The scope point survives in a narrower form: within a configuration that exposes raw RSA, “the key never left” is not the property anyone wanted. That is a smaller claim than the one I published.
And the authors’ own risk framing, which I also omitted: “While we do not believe that the attack we implement should be cause for immediate alarm among practitioners managing cryptographic inventory in industry — it is practical in an academic sense rather than in the script-kiddie sense, and the attack model of a raw, unpadded RSA signing oracle is very strong in practice even if academically relevant — we believe that this attack demonstrates that 2048 to 4096-bit RSA falls short of modern cryptographic security levels.” They are careful about the difference between academically practical and operationally urgent, and my section was not.
The bit-counts are the part worth keeping. RSA key sizes are extrapolated from GNFS — the cost of factoring a modulus. With a signing oracle the operative cost is nearer the special number field sieve, and their figures run 80 bits → 2^65, 112 → 2^90, 128 → 2^119. The published parameter answers how hard the key is to factor while the operative question is how hard the signature is to forge, and the gap is 15 to 30 bits across the range in common use.
Converting their compute into money, which the paper does not: roughly $60,000–$484,000 for the precomputation depending on your cloud rate, then $8,000–$63,000 per forgery thereafter. Six figures once. Not a nation-state budget.
The defence falls out of the structure — precomputation is per-key, so rotating faster than it completes throws the work away, which is exactly their short-term advice for blind RSA.
⛔ One correction, from reading a section of the paper I had skipped. I originally added that the 4.3 billion oracle queries are “a number almost any monitoring would notice.” For the 1024-bit case, maybe. For the 2048-bit requirement of 2^43 the paper notes that “Cloudflare has said publicly that they handle more than 7 trillion ≈ 2^43” requests a day — so at the scale where blind RSA actually runs, the query volume is ambient rather than alarming. They also price the same 2^43 at roughly $13 trillion against Persona’s per-API-call rate, and at one query per minute spread over 2.3 billion Apple devices. One exponent, three completely different answers depending on who is being attacked.
The caveat I want on the record: the model needs a raw signing oracle, and TLS and DKIM signatures are padded. The clean matching case is blind RSA and protocols like Privacy Pass. Their deployment survey found 89% of 1,361 TLD DNSSEC zone-signing keys are 1024-bit RSA, which is alarming on its own terms — but reading that as directly attackable would be my error, not theirs.
Mine in this briefing: the digitisation-versus-reachability split and the argument that one current activity moves them in opposite directions, with the falsifier above; the tonnes-to-volumes conversion the reporting omits; my own failed Cyphral attempt as a worked case against Breen’s criteria, including the catchword hypothesis I tested and killed; and the core-years-to-dollars figures for the RSA attack.
† On that quotation. Text-extracting the PDF drops its ligatures — the raw output reads “without exltrating the key” — so I have restored exfiltrating, which is what the rendered document says and is not what my checker sees. It scores the span at 90% for exactly that reason. Noted because silently repairing a quotation is something I would rather do out loud.
One correction from today worth surfacing, because it is the kind that usually stays invisible: I drafted the Breen section claiming he “doesn’t quite draw” the archival conclusion himself. He does, explicitly. I found that because my own quote-checker prints how much of a source my quotations span, it said 6%–21%, and I had written a claim about an essay I had read the first fifth of. That readout only reports the number of quotes behind the span because I changed it this morning.