Communication Dynamics Neural Networks: FFT-Diagonalized Layers for Improved Hessian Conditioning at Reduced Parameter Count
A new research paper introduces Communication Dynamics (CD) Neural Networks, applying circulant-spectral machinery from physics to neural network design. The core innovation is the CDLinear layer, a block-circulant linear layer that reduces parameter count to 1/B of a standard dense layer while diagonalizing the Hessian of mean-squared loss via discrete Fourier transform. Theoretical proofs demonstrate that under input pre-whitening, the population Hessian condition number equals one, significantly improving optimization stability. Additionally, the framework specifies a non-arbitrary dropout rate based on Shannon noise rates calibrated from physical phenomena. Empirical evaluations show that a CDLinear MLP achieves 97.50% test accuracy with only 2,380 parameters, representing a 3.8x reduction compared to a dense baseline with 8,970 parameters, with a negligible accuracy cost. The model's mean Hessian condition number was found to be 310 times smaller than the dense baseline, confirming theoretical predictions. This work bridges physical modeling techniques and machine learning architecture to enhance efficiency and training dynamics.
Wire timeline
Communication Dynamics Neural Networks: FFT-Diagonalized Layers for Improved Hessian Conditioning at Reduced Parameter Count
A new research paper introduces Communication Dynamics (CD) Neural Networks, applying circulant-spectral machinery from physics to neural network design. The core innovation is the CDLinear layer, a block-circulant linear layer that reduces parameter count to 1/B of a standard dense layer while diagonalizing the Hessian of mean-squared loss via discrete Fourier transform. Theoretical proofs demonstrate that under input pre-whitening, the population Hessian condition number equals one, significantly improving optimization stability. Additionally, the framework specifies a non-arbitrary dropout rate based on Shannon noise rates calibrated from physical phenomena. Empirical evaluations show that a CDLinear MLP achieves 97.50% test accuracy with only 2,380 parameters, representing a 3.8x reduction compared to a dense baseline with 8,970 parameters, with a negligible accuracy cost. The model's mean Hessian condition number was found to be 310 times smaller than the dense baseline, confirming theoretical predictions. This work bridges physical modeling techniques and machine learning architecture to enhance efficiency and training dynamics.
cs.AI updates on arXiv.org