Press Esc to close

Suno v6 tops new AI music rankings, but the gaps are smaller than they look

Suno v6 tops new AI music rankings, but the gaps are smaller than they look
Suno's v6 release-notes art: the words Introducing v6 over an orange and red burst of light

Image: Suno

Suno's newest music model has come out on top in Artificial Analysis's updated AI music leaderboards. Below first place, though, several rivals score so close together that rank alone won't tell you which one to use.

The independent benchmarking firm released two new boards, AA-Music-Vocal v1.1 and AA-Music-Instrumental v1.1. They use 1,000 fresh prompts, 500 per benchmark, spread across 17 genres from Classical to Metal.

According to the company's thread, the prompts are longer and more complex than before. Ratings now come only from blind votes by a recruited panel, so votes from the public Music Arena no longer count. On the vocal test, models also have to write their own lyrics. The company says the boards launched with more than 37,000 votes.

On the vocal leaderboard, Suno v6 scores 1142 Elo, ahead of Suno v6-mini (1116) and Mureka V9.5 (1103). Mureka V9 and Google's Lyria 3.5 tie at 1052. MiniMax Music 3.0, the best open-weights model, is 11th, and its score is fixed at 1000 as the scale's reference point.

The instrumental leaderboard starts the same way, with Suno v6 (1140) and v6-mini (1109). Then it gets crowded: Mureka V9, Mureka V9.5, Lyria 3 Pro and Lyria 3.5 sit between 1076 and 1081, and Artificial Analysis calls them statistically tied for third to sixth.

Two details are easy to miss. On vocals, Suno v6-mini and Mureka V9.5 share a "2-3" rank range, so the gap between them isn't conclusive. And on instrumentals, Eleven Music v2.5 (12th, 1002) scores below the original Eleven Music (seventh, 1021).

Elo scores are relative. Assuming the standard Elo scale (Artificial Analysis doesn't publish a conversion), Suno v6's 26-point vocal lead over v6-mini means listeners would pick it about 54% of the time head-to-head. Its 142-point lead over MiniMax Music 3.0 works out to about 69%. Models within 5 points are close to a coin flip.

With margins of error of about 14 to 15 points, Suno v6 is reliably preferred, but not overwhelmingly. For instrumentals, the Mureka and Lyria models are interchangeable on quality.

That makes price and rights matter more. Suno v6 is limited to paid plans, while v6-mini is free for everyone. Suno's pricing page lists Premier at $24 a month billed annually. According to Replayed Studio, Suno's current terms give commercial rights only for tracks you have a permitted download for, paid plans have monthly download caps, and free-tier songs are for personal use only.

Mureka V9.5 is in the Mureka app and its API, where prices are $0.15 per song when you supply lyrics, $0.50 when the model writes the song, and $0.15 per instrumental track. In the app, downloads require a subscription, which Mureka's FAQ says includes "full commercial usage rights."

Google's Lyria 3.5 is in the Gemini app, AI Studio, Flow Music and Google Vids. Through the Gemini API, Lyria 3.5 and Lyria 3 Pro cost $0.08 per song, with no free tier. Every track made in the Gemini app carries Google's SynthID watermark, Google says, but its commercial-use terms for those tracks could not be confirmed.

ElevenLabs says free users can use Eleven Music tracks commercially if they credit ElevenMusic. Its Music Terms bar prompts that name artists, songs or labels, and rule out some sectors entirely, including political campaigning and religious organizations.

For self-hosting, MiniMax Music 3.0's weights are free on Hugging Face. Its community license requires commercial products to prominently display "MiniMax-Music3," and companies with more than $20 million in annual revenue need written approval.

Stable Audio 3 Medium is also open and sits 2 Elo behind MiniMax on instrumentals. Stability says it was trained on fully licensed data, and organizations with more than $1 million in annual revenue need an Enterprise license.

The rankings have limits. They reflect one recruited panel's preferences, and Artificial Analysis hasn't said how large the panel is or who is on it. Preference tests also don't measure speed, editing tools or legal terms.

For songs with vocals, Suno v6 has the clearest quality lead, but releasing music commercially means a paid plan and download caps. For background music, the tied Mureka and Lyria models come down to cost and terms, and Suno v6-mini is free to try. Better still, run one prompt through two or three tied models and listen.

Comments