Paperlane

no. 16 · Tuesday 17 November 2026

Certified Nash Equilibrium Learning for Nonzero-Sum Jump-Diffusion Games: Policy Iteration and Off-Policy Q-Learning

Pengwei Wang, Na Li, Jiacheng Fan, Tielong Shen and Zuo Quan Xu

math.OC · 31 pp

Abstract

This paper establishes two model-free reinforcement learning methods for infinite-horizon N-player linear-quadratic nonzero-sum stochastic differential games whose drift, Brownian diffusion, and Poisson jump amplitudes depend on all players' controls. Control dependence in the diffusion and jump terms couples the players' simultaneous feedback updates. A verifiable certificate ensures that the partially model-free policy iteration map is contractive on an invariant closed ball centered at an admissible initial feedback profile. All feedback iterates are mean-square stabilizing, and the algorithm achieves local geometric convergence of feedback gains and value matrices to those associated with a closed-loop Nash equilibrium. The term certified refers to a verifiable convergence certificate that guarantees mean-square stability of all policy iterates and local geometric convergence to a closed-loop Nash equilibrium. To implement the policy updates from data, the off-policy Q-learning algorithm reuses the same behavior-policy data across iterations without requiring the system coefficients or jump intensity. Under quantitative persistence of excitation and a common initialization, exact population regression reproduces the policy-iteration sequence. The same local convergence guarantee therefore applies to off-policy Q-learning at the population level. Simulation results demonstrate the effectiveness of the two proposed algorithms.

comments
31 pages, 3 figures
fields
Optimization and Control
pdf ↗ page arXiv:2611.21272 ↗