Paperlane

no. 17 · Wednesday 18 November 2026

Safe Reinforcement Learning with Hard Constraints: Policy Gradient Methods with Intersample Guarantees

Vipul K. Sharma and S. Sivaranjani

math.OC · via eess.SY

Abstract

We develop a policy-gradient (PG) reinforcement learning (RL) framework with hard inter-sample control barrier function (CBF)-based safety constraints for nonlinear sampled-data systems. We first design a class of stochastic policies with support truncated to sampling-period-dependent safe control sets, that guarantee safety-by-construction at sampling instants. We then propose a model-free PG algorithm that searches over the class of safe-by-design policies and establish sufficient conditions on sampling-period such that any action sampled from safe control sets remains safe throughout the sampling interval, providing inter-sample guarantees. We further establish finite-time convergence and sample complexity guarantees for the proposed PG algorithm and characterize its global optimality under gradient dominance. Finally, we illustrate inter-sample safety guarantees on a safe quadrotor navigation environment.

fields
Optimization and Control
pdf ↗ page arXiv:2611.21987 ↗