no. 17 · Wednesday 18 November 2026
Canonical Layer Inference for Deep Gaussian Processes via Symmetry-Breaking Augmentation
Abstract
Deep Gaussian processes (DGPs) are flexible priors for estimating compositional functions, but function-level accuracy does not determine their intermediate-layer representations. The basic obstruction is an adjacent-layer symmetry: applying an invertible transformation and its inverse to two adjacent layers leaves the composed regression map unchanged. Under explicit symmetry and prior-comparability conditions, we show that the posterior may not concentrate on any prescribed intermediate-layer representation, even when the composed regression function is estimable. We introduce a layerwise exponential tilt of the DGP prior, using a penalty proportional to each hidden layer's distance from the identity map, with stronger weights on earlier layers. The methodology links an exact posterior-odds rule along likelihood-preserving symmetries to quantitative control of redundant layers. Under proxy and local prior-mass conditions, we derive a penalty window that preserves function-level fractional posterior contraction while localizing redundant layers at certain radius. The resulting design rule calibrates representation regularization to the function-estimation rate. It favors decompositions with earlier layers close to identity; uniqueness and automatic depth selection are not established. The general theory and methodology are supported with detailed examples.
- fields
- Statistics Theory