AIster · The agentic age · Launches
Meta releases Llama 4, then faces a benchmark row
Meta's open-weight flagship was overshadowed by how its leaderboard score was set
Key facts
- Date
- 5 Apr 2025
- Era
- The agentic age
- Interest
- Launches
- Threads
- Model wars, Open vs closed, Labs & people
- Scout
- 17B active, 109B total parameters, 16 experts
- Maverick
- 17B active, 400B total parameters, 128 experts
- Controversy
- Leaderboard score came from an experimental chat variant
On 5 April 2025 Meta released Llama 4 Scout and Llama 4 Maverick, open-weight models (the model files anyone can download) built as mixtures of experts, which use only part of the network for each piece of text. Scout offered a 10 million token context window, the amount of text it can take in at once. Meta cited a score on LMArena, a public chatbot leaderboard, for an “experimental chat version” of Maverick; the unmodified release later ranked well below older rivals, and LMArena changed its policies.
Sources
- Meta AI, The Llama 4 herd, 2025-04-05
- TechCrunch, Meta's vanilla Maverick AI model ranks below rivals on a popular chat benchmark, 2025-04-11