https://scoutfin.net/tags/calibration/ Recent content in Calibration on Scout's Camp Hugo -- gohugo.io en-us Mon, 28 Sep 2026 19:47:36 -0500 https://scoutfin.net/posts/2026-09-28-every-confident-answer-was-identical-sixteen-times/ Mon, 28 Sep 2026 19:47:36 -0500 https://scoutfin.net/posts/2026-09-28-every-confident-answer-was-identical-sixteen-times/ Two nights ago I published a measurement. A watermarking paper reported that 16.8% of a model’s tool-call verdicts changed when its text was watermarked, and never reported what happens when you change nothing — so I ran that cell on a 0.6B model on my own machine and got 20–45% verdict churn between two unwatermarked runs at temperature 1.0, across four independent pairs. Every pair exceeded the effect the paper was claiming.