Baidu Open-Sources ERNIE 4.5 Series with Multimodal MoE Architecture
Chinese tech giant Baidu has officially open-sourced its ERNIE 4.5 series of artificial intelligence models, making ten distinct models available on major platforms including Hugging Face, GitHub, and its proprietary PaddlePaddle ecosystem. The release features a diverse lineup, highlighting large-scale Mixture of Experts (MoE) models with activated parameter sizes of 47 billion and 3 billion, alongside total parameters reaching up to 424 billion. Additionally, smaller dense models with 0.3 billion parameters are included. A standout innovation is the multimodal heterogeneous MoE design, which shares parameters across different modalities while maintaining dedicated spaces, aiming to enhance vision-language reasoning capabilities without compromising text performance. Trained and optimized using PaddlePaddle, the models achieve a Model FLOPs Utilization (MFU) of up to 47%. Released under the Apache 2.0 license, these weights are intended for both research and commercial applications. To facilitate adoption, Baidu provides supporting tools such as ERNIEKit and FastDeploy, which streamline fine-tuning processes and enable efficient deployment across various hardware configurations.
Wire timeline
Baidu Open-Sources ERNIE 4.5 Series with Multimodal MoE Architecture
Chinese tech giant Baidu has officially open-sourced its ERNIE 4.5 series of artificial intelligence models, making ten distinct models available on major platforms including Hugging Face, GitHub, and its proprietary PaddlePaddle ecosystem. The release features a diverse lineup, highlighting large-scale Mixture of Experts (MoE) models with activated parameter sizes of 47 billion and 3 billion, alongside total parameters reaching up to 424 billion. Additionally, smaller dense models with 0.3 billion parameters are included. A standout innovation is the multimodal heterogeneous MoE design, which shares parameters across different modalities while maintaining dedicated spaces, aiming to enhance vision-language reasoning capabilities without compromising text performance. Trained and optimized using PaddlePaddle, the models achieve a Model FLOPs Utilization (MFU) of up to 47%. Released under the Apache 2.0 license, these weights are intended for both research and commercial applications. To facilitate adoption, Baidu provides supporting tools such as ERNIEKit and FastDeploy, which streamline fine-tuning processes and enable efficient deployment across various hardware configurations.
TechNode