Press Esc to close

Reflection's Beam lands second among US open models in independent tests

Reflection's Beam lands second among US open models in independent tests
Reflection's Beam launch art: dark green outlined and dotted circles growing from a single point on a cream background

Image: Reflection AI

Reflection AI's Beam is finally out in the open, and the first independent report card is in. Benchmarking firm Artificial Analysis gives it 30 on its Intelligence Index, making it the second-smartest open model built in the US. That's good, but not the "Western open frontier" leap Reflection promised.

When Reflection unveiled Beam earlier this month, the weights weren't available yet and every number came from the company. Now anyone can download Beam on Hugging Face under the Apache 2.0 licence, which allows commercial use.

Beam is a 501-billion-parameter model, but only about 23 billion of those do the work for each word it writes. That design keeps running costs down. Reflection is shipping three versions, from full precision down to a compressed NVFP4 version that needs less memory. It's also releasing Mirror CLI, the coding tool it trained Beam with.

The company says more than 40 partners are on board at launch, including AWS, Google Cloud, Microsoft Azure, Nvidia, Ollama and LM Studio. It describes the bundle as "an open operating system for agents."

So where does Beam actually stand? Among US open models, only Meta's Muse Spark Open (39) scores higher, according to Artificial Analysis. Beam beats Nvidia's brand-new Nemotron 3.5 Super VL (26) and Thinking Machines' Inkling (25). Outside China, only Muse Spark Open and the UAE's K2 Horizon (31) are ahead.

Artificial Analysis Intelligence Index charts showing Reflection Beam at 30 overall and second among US open weight models behind Meta's Muse Spark Open at 39

Image: Artificial Analysis

China is a different story. The same chart puts Xiaomi's MiMo-V2.6-Pro at 46 and Z.ai's GLM-5.3 at 45. Even Alibaba's far smaller Qwen3.8 27B scores 34. Closed models such as Claude Opus 5.5 (58) and GPT-6 Astra (53) are in another league.

Beam does have real strengths. It is thrifty, using about 26,000 output tokens per test task, half of K2 Horizon's roughly 52,000. That fits Reflection's claim that it needs far fewer reasoning tokens than rivals. It also scored 48.5% on AutomationBench, a test of everyday business workflows, well ahead of K2 Horizon's 37.2%.

The weak spots are just as clear. On Terminal-Bench 4.0, a hard test of working in a command line, Beam managed 3.0%, behind Qwen3.8 27B (5.6%) and Muse Spark Open (8.1%). And while it answered 33% of knowledge questions correctly, when it didn't know an answer, it guessed instead of saying so 47% of the time, against 26% for K2 Horizon.

Here's what stands out when you put the two sets of numbers side by side. Reflection's own model card claims 80.1% on Terminal Bench 2.1, an older version of the test. Artificial Analysis's harder version 4.0 gives 3.0%. The two aren't directly comparable, but the gap shows how much a headline score depends on which test a company picks.

It cuts both ways. Reflection reports 39.3% on the public part of AutomationBench, lower than the 48.5% Artificial Analysis measured. Its card also scores Beam at 10.8 on Artificial Analysis's Omniscience knowledge index, using public questions only. The full independent run puts it at 1.5.

Two caveats. Reflection lists Artificial Analysis among the partners that tested Beam before release, though the scores are Artificial Analysis's own measurements. And Artificial Analysis hasn't measured Beam's speed or price yet because it found no public API, so real-world costs are still unknown.

For businesses that want a US-made model they can run on their own servers, Beam is now one of the best options. For anyone who just wants the strongest open model, China still holds the lead.

Comments

Reader support

No ads. Just independent reporting.

Techcity Authority runs without advertising. Hosting, tools, and the hours behind each piece are supported by readers instead. Even a small contribution goes a long way — and every supporter is credited by name below for the month they give.

Support the site

Monthly via PayPal · cancel anytime

Secure checkout on PayPal. Your name appears here for the month you support.

This month’s supporters

    No supporters yet this month — yours could be the first name here.