Baidu officially open-sourced the ERNIE 4.5 Series, releasing ten models on Hugging face, GitHub, and its PaddlePaddle eco-system. The lineup includes large-scale MoE models with activated parameters sizes of 47 billion to 3 billion (total parameter up to 424 trillion), as well as smaller dense models of 0.3 billion parameters. The multimodal heterogeneous MoE is a key feature. It shares parameters between modalities, while reserving spaces for text. This design is intended to enhance tasks such as vision-language reasoning, without sacrificing text performance. The models were optimized and trained using PaddlePaddle. Baidu reported MFU (model FLOPs usage) of up to 47 percent. Weights are released as Apache 2.0 and are intended for research and commercial purposes. Supporting tools such as ERNIEKit, FastDeploy and Multi-Hardware Deployment simplify fine-tuning.[Baiduin Chinese]
