Paperlane

no. 17 · Wednesday 18 November 2026

Neural Classifiers as Error-Correcting Codes

Sergey Salishev, Anton Makarov, Oleg Granichin, Renata Avros and Zeev Volkovich

math.IT · v2 · 43 pp

Abstract

A unified framework is proposed that characterizes supervised deep classification as the decoding of error-correcting codes in the Shannon sense. Cross-entropy optimization is shown to align softmax classifiers with maximum-likelihood decoding, where logits map to Hamming distances to class-specific codewords. This correspondence explains the emergence of standard divergence objectives (e.g., Kullback-Leibler, Jeffreys, Bregman) and provides a geometric basis for the stability of decision boundaries. Furthermore, the geometries of Neural Collapse and LeJEPA are identified as optimal constellations for decoding under isotropic noise. Unlike classical Error-Correcting Output Codes, deep networks are shown to learn effective codebooks in the feature space implicitly. This insight is leveraged to derive novel criteria for lossless quantization, improved knowledge distillation, and architectural regularization based on code redundancy. Finally, this perspective is extended to self-supervised encoders, offering a cohesive explanation for the geometry of latent representations in modern neural models.

comments
43 pages, 6 figures. v2: author accepted manuscript; includes published-version DOI link and CC BY-NC-ND 4.0 notice. Published in Information Sciences 766 (2027) 124394
journal
Information Sciences 766 (2027) 124394
fields
Information Theory
pdf ↗ page arXiv:2611.18239 ↗