Paperlane

no. 16 · Tuesday 17 November 2026

Equilibrium measures for anisotropic exponential interaction energies with applications to transformers

Oleksii Kachaiev, Martin Burger, Samira Kabri, Yury Korolev and Tim Roith

math.AP · 23 pp

Abstract

We study the minimization of anisotropic exponential interaction energies over probability measures on the sphere, motivated by mean-field models of self-attention in transformer architectures. For every symmetric positive definite interaction matrix, we relate the minimizer to a corresponding anisotropic Brownian motion and use the Feynman–Kac formula and elliptic regularity theory to show that it admits a smooth, strictly positive density on the sphere. We further show that it is the only stationary measure with full support for the dynamics considered. We use the stochastic viewpoint to derive an algorithm for sampling from the minimizer. In dimension two, we also propose a numerical scheme for approximating its density based on a Mathieu series expansion.

comments
23 pages, 3 figures
fields
Analysis of PDEs, Probability
pdf ↗ page arXiv:2611.19518 ↗