Press Esc to close

Baidu open-sources ERNIE 5.5 Flash Base, a 280B model with 14B active

Baidu open-sources ERNIE 5.5 Flash Base, a 280B model with 14B active
ERNIE 5.5 Flash Base architecture

Image: Baidu

Baidu released ERNIE 5.5 Flash Base, an open-weight language model with 280 billion total parameters and 14 billion active per token. The release is the base model, not a chat model. It is the checkpoint at the end of pre-training, with no instruction tuning or reinforcement learning applied, and Baidu says it is meant as a starting point for developers who want to fine-tune their own models.

The model was pre-trained from scratch on 33.5 trillion tokens, with a training context of up to 128,000 tokens and support for a 1 million token context during fine-tuning and inference. It is a sparse mixture-of-experts design, so only a fraction of its parameters activate for each token. Baidu says it matches state of the art at its size, though that is the company's own claim.

The architecture is the main story. Most of the model's layers compress every 128 tokens into a single cached entry, while one layer in seven keeps full token-level access. Baidu says that keeps the KV cache small without losing long-context performance. The model also uses several independent routers for its mixture-of-experts layer, a 4-stream residual connection, a piecewise output temperature and a reuse of the final layer to predict future tokens. None of that is visible to a user, but it is the reason a 280B model can run with 14B active parameters.

ERNIE 5.5 Flash Base benchmark comparison

Image: Baidu

On benchmarks, Baidu compared ERNIE 5.5 Flash Base against DeepSeek-V4-Flash-Base, Hy3-preview-Base and MiMo-V2.5-Base, all evaluated with the same harness. It scored strongly across 30 benchmarks in seven categories, with reasoning, code and exam tasks among its best results. The six full-context layers were enough to keep long-context scores up, even though 37 of the 43 layers work on heavily compressed input. These are Baidu's own runs, not independent verification.

The weights are on Hugging Face in FP8 and BF16 safetensors, under Baidu's own license agreement. The model expects a BOS token at the start of every input, so developers should keep add_special_tokens=True when tokenizing. SGLang and vLLM serve it through the completions API.

The honest caveat is that this is a base model. It has not been through instruction tuning or reinforcement learning, so it is not a drop-in replacement for ChatGPT or Claude. It is a foundation that other teams can fine-tune. Baidu says the post-trained ERNIE 5.5 Flash will follow, which is the version most users would actually run.

Baidu's real bet is efficiency at scale. A 280B model that activates 14B parameters per token, with a compact KV cache, is cheaper to serve than a dense model of the same size. Whether that holds up outside Baidu's own benchmarks is the next question.

Comments

Reader support

No ads. Just independent reporting.

Techcity Authority runs without advertising. Hosting, tools, and the hours behind each piece are supported by readers instead. Even a small contribution goes a long way — and every supporter is credited by name below for the month they give.

Support the site

Monthly via PayPal · cancel anytime

Secure checkout on PayPal. Your name appears here for the month you support.

This month’s supporters

    No supporters yet this month — yours could be the first name here.