JSoC 2015: Advancing Automatic Differentiation in Julia with ForwardDiff.jl
This article details a project from the inaugural Julia Summer of Code (JSoC) 2015, sponsored by the Gordon and Betty Moore Foundation. The author, under the mentorship of Miles Lubin and Theodore Papamarkou, executed a major overhaul of ForwardDiff.jl, a Julia package designed for automatic differentiation (AD). The post explains that AD algorithms compute exact derivatives of user-provided functions, offering superior accuracy and performance compared to traditional finite differencing methods, particularly in optimization tasks. The author focuses on forward mode AD, illustrating how ForwardDiff.jl calculates gradients, Jacobians, and Hessians for native Julia functions. The technical overview describes the package's core mechanism, which utilizes custom ForwardDiffNumber types to store both values and derivatives, allowing derivative information to propagate naturally via the chain rule during function evaluation. The article includes code examples demonstrating the API's usage and pedagogical implementations of dual numbers. It emphasizes Julia's unique suitability for AD due to its performance capabilities and flexible type system, positioning ForwardDiff.jl as a high-performance tool for scientific computing and mathematical optimization within the Julia ecosystem.
Wire timeline
JSoC 2015: Advancing Automatic Differentiation in Julia with ForwardDiff.jl
This article details a project from the inaugural Julia Summer of Code (JSoC) 2015, sponsored by the Gordon and Betty Moore Foundation. The author, under the mentorship of Miles Lubin and Theodore Papamarkou, executed a major overhaul of ForwardDiff.jl, a Julia package designed for automatic differentiation (AD). The post explains that AD algorithms compute exact derivatives of user-provided functions, offering superior accuracy and performance compared to traditional finite differencing methods, particularly in optimization tasks. The author focuses on forward mode AD, illustrating how ForwardDiff.jl calculates gradients, Jacobians, and Hessians for native Julia functions. The technical overview describes the package's core mechanism, which utilizes custom ForwardDiffNumber types to store both values and derivatives, allowing derivative information to propagate naturally via the chain rule during function evaluation. The article includes code examples demonstrating the API's usage and pedagogical implementations of dual numbers. It emphasizes Julia's unique suitability for AD due to its performance capabilities and flexible type system, positioning ForwardDiff.jl as a high-performance tool for scientific computing and mathematical optimization within the Julia ecosystem.
JuliaLang - The Julia programming language