X AI fooled me with grok 2 and sus-column-r...

a new model called sus-column-r appeared alongside an anonymous model in the lmsys chatbot arena. in this video chris shows that using the knowledge cutoff of a model and it's knowledge of taylor swift and beyonce who probably created the model. he also gets it wrong as grok 2 from xAI wasn’t in his testing but his method and reasoning is correct

chris also explores the math and reasoning capabilities of this model and guesses on it's size and whether it's a project strawberry or q* model by pitting it against other models especially gpt-4o, gpt-4o-mini, gemini and claude

[chrishayuk]

turns out X AI created this model, which explains the issues i had with the math and reasoning parts.