The Case for Dedicated Programming Languages in Machine Learning

This article analyzes the intersection of machine learning (ML) and programming language (PL) theory, arguing that modern ML frameworks function as distinct, albeit non-standardized, programming languages. While tools like TensorFlow and PyTorch utilize Python interfaces, they effectively create separate runtimes and expression trees, leading to complex metaprogramming overheads. The text highlights a critical tension: general-purpose languages like Python offer flexibility but struggle with the performance, parallelism, and compilation requirements of advanced ML models. Conversely, static graph-based approaches optimize performance but lack the dynamism needed for research involving recursion and control flow. Hybrid solutions like MXNet’s Gluon attempt to bridge this gap but often result in fragmented APIs. The author suggests that the future of ML lies in developing dedicated languages or flexible runtime systems that support programmable semantics, automatic differentiation, and efficient deployment. This approach aims to balance the need for high-performance numerical computation with the expressive power required for innovative model architectures, moving beyond current library-based limitations.