no. 17 · Wednesday 18 November 2026
Multi-Stage Stochastic Optimization and Reinforcement Learning Approaches for Dynamic Inspection of Infrastructure Systems
Abstract
We study a dynamic inspection problem for infrastructure systems in which we route and schedule multiple vehicles to monitor components that are subject to heterogeneous stochastic failures. Inspection decisions determine which components the vehicles serve first and thus affect how failures propagate to neighboring components. We formulate the problem as a multi-stage stochastic mixed-integer program that jointly optimizes routing and scheduling under exogenous and decision-dependent uncertainty. We develop a stochastic dual dynamic integer programming (SDDiP) algorithm based on a stage decomposition of the multi-stage model and show that it converges finitely to the optimal value. In parallel, we propose a reinforcement learning (RL) approach that learns assignment policies from simulated system dynamics. On instances built from the road and storm-sewer networks of Sioux Falls, SDDiP proves optimality where the extended formulation of the same model leaves large gaps. On larger instances, RL with a routable-assignment certificate often finds cheaper policies than SDDiP under a short computational budget, whereas SDDiP finds cheaper policies and bounds on their quality with more time. Moreover, modeling failure propagation and decision-dependent transition probabilities reduces the realized cost when these mechanisms are present. These comparisons guide the choice between optimization and policy learning for inspection planning under failure risk.
- comments
- 44 pages, 9 figures
- fields
- Optimization and Control