The melting temperature of a next-token distribution
Softmax sampling is the Boltzmann distribution, so a language model can melt. I found its melting temperature in closed form and tested it on 12 models.
- Predicted from the logit gap and log V alone, across a 976x range of vocabulary sizes
- 8 of my hypotheses died, and all 8 are published
October 2026Most of my ideas here were wrong, and the experiments said so loudly. I kept every failed hypothesis in the release because they taught me the most, and the whole thing ran on a laptop with no GPU.
