Microsoft Promises a 'Whale' for GPT-5, Anthropic Delves Inside a Model’s Mind and Altman Stumbles

Microsoft promise ‘whale-size’ compute for a GPT-5-tier model, and say the end is not in sight for scaling the power of AI. Google ship models and a fascinating paper, while Anthropic unveil the inner workings of large language models. Meanwhile Sam Altman is forced to repeatedly apologize, Ilya Sutskever leaves, and GPT-4o is pushed back. My reflections on all of the above, and details you may have missed from each paper.

[RedBatRacing]

We're going to run out of marine animals real quick. They should have started with plankton

[thallaasalwolf]

Hope anthropic names their next model harpoon

[AI_native]

_- "Our mission is to ensure that artificial general intelligence benefits all of humanity"_
- Literally partners with Rupert Murdoch

[AlexanderMoen]

the Anthropic bot calling itself deplorable and recommending it get deleted from the Internet is super interesting. It makes superalignment seem at least plausible

[marcostrujillo]

Is it only me who feels that the Antrophic paper is a watershed? I mean, as someone who has studied his neurobiology is evident that something akin to even this very limited "SAE" model for the human brain would be deemed as a HUGE breakthrough in neuroscience. Obviously, this is only a proof of concept, and if it can actually be refined and perfected the implications (positive and negative) are almost self-evident.

[damienhughes]

Did anyone notice that there's currently a Golden Gate Claude version available for a limited time? If you go to Claude and look in the upper right, there's a Golden Gate Bridge icon. If you click on it, you can talk to the altered state version referenced in Claude's tinkering with the model research paper. It's crazy!

[timwang]

The Anthropic research paper findings are some of the craziest discoveries I've ever seen in the AI domain.

[GoldenBeholden]

Your section on Anthropic's mapping of their model is the most interesting thing you've talked about on this channel -- and that's saying a lot. Such insights and control open up more possibilities in my opinion than just scaling compute and data.

[facts]

5:45 “Gemini 1.5 pro doesn’t have the rizz of gpt-4o” isn’t something I thought I would hear you say😂

[ryzikx]

anthropic once again showing why they are the leaders in ai safety

[NitFlickwick]

Computerphile recently had an episode talking about a paper discussing the shapes of the curve of the efficacy of future training and that being logarithmic instead of exponential due to lack of data. I’d love to hear your take on that paper.

[romanpfarrhofer]

Ad this whale comparison:
Americans will measure with anything but the metric system

[Rawi]

"Deeply deceptive Ai that hated itself" ✌️😔 real bot

[jeff__w]

15:02 “That’s a pretty abstract concept, right? Making an error in code.”
I dunno—I don’t think it’s any more abstract than, say, the concept of, say, a mistake in grammar, which these language bots are pretty good at detecting.

17:17 “It suggested an internal conflict of sorts.”
18:18 “It sheds light on the concepts the model uses to construct an internal representation of its AI character.”
I tend to find statements like these a little jarring, especially in connection with these language models. I wouldn’t say that’s an “internal conflict”—which suggests some psychological drama roiling under the surface. It’s simply two different verbal outputs that are possible, given the training data—you ramp up “the feature related to hatred and slurs to 20x its maximum activation value, ” get hatred and slurs (no surprise there), and then get the verbal output that might follow what was just said. (A person who has just had a hate-filled outburst might follow it with a similar contrite, self-punishing response.) If there is any “self-hatred” going on with these AI models, I’d be _really_ surprised.

And these models don’t _have_ “internal representations” of their AI character or anything else. (That’s an, to me, unfortunate carry-over from cognitive psychology.) Maybe some people would consider the word embeddings to be “representations” but I wouldn’t consider a list of features, no matter how extensive, of, say, a cat, to be a “representation” of that cat. It has, at best, weights and biases, which give rise to some verbal output when asked to describe its AI character. To me, it’s just muddy wording on the part of the people creating these models, which gets in the way of analysis.

[GiedriusMisiukas]

0:00 (!)
5:56 Math, thought, contemplation
9:51 AI impact on photography art and industry
12:47 on undrerstanding Anthropic LLM inner workings. #monosemanticity
18:24 on AI deceptiveness
22:30 on the voice similarity to Scarlett’s from the movie “Her”

[CyberSQUID]

Anthropics work to understand the models will give them a significant advantage

[epg-]

That Claude response makes me think we should hold off on giving these things full agency inside a robotic body until we have a much better grasp on what's actually going on in their minds. The last thing we want is a robot hearing some words it doesn't like, then deciding that whoever said them should be eliminated.

[TheEtrepreneur]

props for the links with fun/relevant captions, first time I've seen a non boring link description. Keep it coming!

[_ptoni_]

thanks god we have you to explain that anthropic paper. was waiting for this one more than anything 😅

[rickandelon]

You are unparalled. Your intellect and insights are a blessing to navigate the difficulties of the AI landscape.