Paperlane

no. 16 · Tuesday 17 November 2026

Mean-field Token Steering

Md Zulfiqur Haider, Asmaa Eldesoukey, Abhishek Halder and Yongxin Chen

math.OC

Abstract

We consider stochastic token steering through a deep stack of transformer layers as a mean-field control problem for an interacting particle system on the unit sphere, where interactions arise from the attention mechanism. In the mean-field limit, the token population evolves via a controlled integro-partial differential equation where the feedback control acts as a corrective vector field guiding the population toward an aggregate configuration of tokens in the representation space. We deduce the minimum effort control to accomplish this task over a fixed time horizon interpreted as normalized layer depth. We show that the resulting controlled dynamics for the token population can be understood as a mean-field Schrödinger bridge adapted to the sphere geometry. Building on this, we propose a novel generalization of the dynamic Sinkhorn recursion to compute the consistent token evolution. Numerical experiments confirm that the proposed approach achieves exact steering of the token population.

fields
Optimization and Control
pdf ↗ page arXiv:2611.19163 ↗