GSoC 2018: Enhancing Flux with Faster Convolutions and New Models
During Google Summer of Code 2018, a developer focused on improving the computer vision capabilities of Flux, a machine learning library for the Julia programming language. The project aimed to add newer models to the Flux model-zoo, implement new features, and significantly optimize the performance of existing layers. Key achievements included an 18-fold speed increase for convolutions and a threefold improvement for BatchNorm operations. Technical improvements involved adding efficient CUDNN bindings for convolutions and BatchNorm, enabling depthwise convolutions on CPU via NNlib, and updating the Metalhead package with new models like VGG and Inception Nets. The work also resulted in the development of several new packages, including FastStyleTransfer.jl and DeepDream.jl. These updates addressed bottlenecks such as broadcasted bias addition by leveraging specific CUDNN functions. While some pull requests were merged, others required updates for compatibility with Julia 1.0. This technical report highlights significant performance optimizations and feature expansions within the Julia deep learning ecosystem.
Wire timeline
GSoC 2018: Enhancing Flux with Faster Convolutions and New Models
During Google Summer of Code 2018, a developer focused on improving the computer vision capabilities of Flux, a machine learning library for the Julia programming language. The project aimed to add newer models to the Flux model-zoo, implement new features, and significantly optimize the performance of existing layers. Key achievements included an 18-fold speed increase for convolutions and a threefold improvement for BatchNorm operations. Technical improvements involved adding efficient CUDNN bindings for convolutions and BatchNorm, enabling depthwise convolutions on CPU via NNlib, and updating the Metalhead package with new models like VGG and Inception Nets. The work also resulted in the development of several new packages, including FastStyleTransfer.jl and DeepDream.jl. These updates addressed bottlenecks such as broadcasted bias addition by leveraging specific CUDNN functions. While some pull requests were merged, others required updates for compatibility with Julia 1.0. This technical report highlights significant performance optimizations and feature expansions within the Julia deep learning ecosystem.
JuliaLang - The Julia programming language