NeuralNetDiffEq.jl: A Neural Network Solver for ODEs in Julia

This article details the development of NeuralNetDiffEq.jl, a software package created during Google Summer of Code 2017 to solve Ordinary Differential Equations (ODEs) using neural networks within the Julia programming language. The project aims to provide an alternative solver that leverages parallelism in time, offering advantages over traditional iterative methods. Based on Lagaris et al.'s 1997 research, the solver utilizes the Universal Approximation Theorem, training neural networks to minimize a loss function defined by the difference between the network's derivative and the ODE's true derivative. Implemented using DifferentialEquations.jl for the interface and KNet.jl for neural network operations, the project combines mathematical theory with machine learning techniques. The author highlights significant challenges, particularly in achieving convergence for systems of ODEs over longer time domains, which required extensive hyperparameter tuning. While the initial implementation supports customized interpolation based on neural network predictions, predicting solutions for larger domains remains an ongoing challenge. The work is publicly available on GitHub, representing a fusion of scientific computing and deep learning methodologies.