Paperlane

no. 12 · Tuesday 10 November 2026

Discrete-Time Zero-Sum Markov Games with Control, Stopping, and State-Dependent Discounting

Guanghui Feng and Xin Guo

math.OC · 29 pp

Abstract

We study a discrete-time two-player zero-sum stochastic game with control and stopping, where both players can stop the game, the discount factors depends on the states, and the running and terminal rewards may be unbounded. A contraction argument for the constrained Shapley equation establishes the existence of the value and a Nash equilibrium consisting of stationary controls and stopping times. We further provides algorithms for approximating the value and computing an $\varepsilon$-Nash equilibrium. A numerical example with state-dependent discount factors illustrates the results.

comments
29pages, 1figure
fields
Optimization and Control
pdf ↗ page arXiv:2611.10396 ↗