The boardAI, LLMs, ML and cognitive neuroscience

The melting temperature of a next-token distribution

The melting temperature of a next-token distribution

Softmax sampling is the Boltzmann distribution, so a language model can melt. I found its melting temperature in closed form and tested it on 12 models.

  • Predicted from the logit gap and log V alone, across a 976x range of vocabulary sizes
  • 8 of my hypotheses died, and all 8 are published

October 2026Most of my ideas here were wrong, and the experiments said so loudly. I kept every failed hypothesis in the release because they taught me the most, and the whole thing ran on a laptop with no GPU.