Fitting Multilinear Polynomials for Logic Gate Networks
A new research paper submitted to arXiv introduces a novel approach to training learnable logic gate networks by fitting multilinear polynomials. The study addresses limitations in the baseline Soft-Mix method, which suffers from vanishing backward signals and interaction-coefficient starvation due to the rank-4 nature of the 16 Boolean gate codebook. The authors propose using a 4-dimensional polynomial space, reducing neuron parameters from 16 to 4, and introduce a Covariance Jacobian (CovJac) method for soft vector quantization. This technique couples starved coefficients to active constant channels, bypassing gradient issues inherent in Straight-Through Estimation (STE). Experimental results across seven datasets demonstrate that the proposed 4-parameter methods match or exceed Soft-Mix performance. Notably, while Soft-Mix collapses significantly in deep networks (e.g., -37.3% accuracy drop on CIFAR-10 at 12 layers), the CovJac method remains stable with minimal performance loss. This work contributes to efficient neural architecture design and combinational circuit learning within machine learning and artificial intelligence domains.
Wire timeline
Fitting Multilinear Polynomials for Logic Gate Networks
A new research paper submitted to arXiv introduces a novel approach to training learnable logic gate networks by fitting multilinear polynomials. The study addresses limitations in the baseline Soft-Mix method, which suffers from vanishing backward signals and interaction-coefficient starvation due to the rank-4 nature of the 16 Boolean gate codebook. The authors propose using a 4-dimensional polynomial space, reducing neuron parameters from 16 to 4, and introduce a Covariance Jacobian (CovJac) method for soft vector quantization. This technique couples starved coefficients to active constant channels, bypassing gradient issues inherent in Straight-Through Estimation (STE). Experimental results across seven datasets demonstrate that the proposed 4-parameter methods match or exceed Soft-Mix performance. Notably, while Soft-Mix collapses significantly in deep networks (e.g., -37.3% accuracy drop on CIFAR-10 at 12 layers), the CovJac method remains stable with minimal performance loss. This work contributes to efficient neural architecture design and combinational circuit learning within machine learning and artificial intelligence domains.
cs.AI updates on arXiv.org