\n<\/p>\n
Earlier this week, Meta landed in hot water<\/a> for using an experimental, unreleased version of its Llama 4 Maverick model to achieve a high score on a crowdsourced benchmark, LM Arena. The incident prompted the maintainers of LM Arena to apologize<\/a>, change their policies, and score the unmodified, vanilla Maverick.<\/p>\n Turns out, it\u2019s not very competitive.<\/p>\n The unmodified Maverick, \u201cLlama-4-Maverick-17B-128E-Instruct,\u201d was ranked below models<\/a> including OpenAI\u2019s GPT-4o, Anthropic\u2019s Claude 3.5 Sonnet, and Google\u2019s Gemini 1.5 Pro as of Friday. Many of these models are months old.<\/p>\n The release version of Llama 4 has been added to LMArena after it was found out they cheated, but you probably didn\u2019t see it because you have to scroll down to 32nd place which is where is ranks pic.twitter.com\/A0Bxkdx4LX<\/a><\/p>\n \u2014 \u03c1:\u0261e\u03c3n (@pigeon__s) April 11, 2025<\/a><\/p>\n<\/blockquote>\n Why the poor performance? Meta\u2019s experimental Maverick, Llama-4-Maverick-03-26-Experimental, was \u201coptimized for conversationality,\u201d the company explained in a chart published<\/a> last Saturday. Those optimizations evidently played well to LM Arena, which has human raters compare the outputs of models and choose which they prefer.<\/p>\n\n