Sparse-Aware Neural Networks for Nonlinear Functionals: Mitigating the Exponential Dependence on Dimension

Researchers have proposed a new framework utilizing sparse-aware neural networks to address challenges in learning operators over infinite-dimensional function spaces. Published on arXiv, this study tackles the curse of dimensionality and limited interpretability inherent in existing deep learning theories for functional learning. The authors introduce a hybrid architecture combining convolutional networks for extracting sparse features from finite samples with deep fully connected networks to approximate nonlinear functionals effectively. By employing universal discretization methods, the team demonstrates that sparse approximators allow for stable recovery from both deterministic and random discrete samples. This approach yields improved approximation rates and significantly reduces the required sample sizes across various function spaces, particularly those characterized by fast frequency decay and mixed smoothness. The findings offer crucial theoretical insights into how sparsity can alleviate exponential dependence on dimension, marking a significant advancement in the efficiency and stability of operator learning models within machine learning and functional analysis.