Joe Carlsmith - Preventing an AI Takeover

Chatted with Joe Carlsmith about whether we can trust power/techno-capital, how to not end up like Stalin in our urge to control the future, gentleness towards the artificial Other, and much more.

Enjoy!

Links:

Timestamps:
00:00:00 - Understanding the Basic Alignment Story
00:44:04 - Monkeys Inventing Humans
00:46:43 - Nietzsche, C.S. Lewis, and AI
1:22:51 - How should we treat AIs
1:52:33 - Balancing Being a Humanist and a Scholar
2:05:02 - Explore exploit tradeoffs and AI

Sponsors:

[michaelzeer]

Remarkable listening to these meaningless ramblings which define nothing and are absent any predictive powers. I began to hope that AI would take over and kill us, if only to end this one-way conversation.

[JeffreyOlmo-mfei]

The only reason Joe isn’t who Dwarkesh was referring to as his biggest guest ever is that he was already on the pod

[fromduskuntodawn]

Count the number of “kind of”s…. I’ll wait.

[iconoclasting]

Dwarkesh is the best.
This was literally hand-waving horse shit. Bring back Ilya or Hinton

[everettvitols]

Dwarkesh I'm sorry that these Youtube comments are slop.

[Terrahpin]

Dont really find these sorts of ai talks particularly insightful.

[odiseezall]

stop anthropomorphizing the space of possible minds please

[Telencephelon]

Wow. The hand movement atr 2x is anywhere between wild to nauseating to scary.

[nokar]

Is AI alignment much different from human alignment? The "Other" framing applies just as well. You're going to need transparency and/or proofs to be able to rely on incentive alignment.

The nice thing with AIs is we don't have to worry about being too invasive, which when working with other humans is of course a difficult tradeoff. I hope we see some more interesting experiments being run in the coming years, but I don't think it should be too far removed from human alignment. AI alignment without human alignment is a powder keg, anyway.

[esterhammerfic]

One of the few guests who seems to take alignment seriously

[sldf]

Joe’s predictions (at least what I could make of them in this very difficult to follow conversation) seemed at odds with the way in which the largest and most effective frontier models are actually being built and trained. DK pushed back on this a little in the first half, but didn’t seem to go deep enough into the technical side to in/validate the basis of Joe’s assumptions.

[Greg-xiyx]

I’m in the David Shapiro camp as far as alignment meaning I don’t think there’s any reason to think alignment will be at all difficult to solve. Full blown acceleration is sentient life’s best chance.

[nicholas]

Forget taking over the world, it would be great if AI could solve autofocus

[mrpicky]

if you close your eyes you might think that Dwarkesh is talking to himself

[jordanarel]

58:31 Umm… Did Dwarkesh just call his sponsor the “paper-clipper“ of the payments industry???

[elisemiller]

Can't really even enter in here, the speakers sound so frenetic in their delivery...literally got to 1 min. 49 secs! Geez!

[cece]

it needs to realize the value of all life!!!

[juliandunn]

Why do both of these men smile awkwardly while trying to communicate?

[squamish]

Dwarkesh, you're great but you gotta slow down.

[jamesfehlinger]

48:27

"What Lewis is anticipating. . .is something like the culmination of the project
of scientific modernity. . . this process of increased understanding of. . .
of the natural environment and a corresponding increase in our ability to control
and direct that environment. . . Naturalism says that humans too. . .
are a part of nature. Insofar as this process of scientific modernity involves. . .
progressively greater. . . ability to control nature, that will presumably grow
to encompass our own natures and the natures of other beings we could create. . .
Lewis views this as a kind of cataclysmic event and crisis. In particular, he believes
that it will lead to all kinds of tyrannical behaviors and attitudes. . ."

'At first, of course, ' said Filostrato, 'the power will
be confined to a number -- a small number -- of
individual men. Those who are selected for
eternal life.'

'And you mean, ' said Mark, 'it will then be extended
to all men?'

'No, ' said Filostrato. 'I mean it will then be reduced
to one man. You are not a fool, are you, my young
friend? All that talk about the power of Man over
Nature -- Man in the abstract -- is only for the _canaglia_.
You know as well as I do that Man's power over Nature
means the power of some men over other men
with Nature as the instrument. There is no such thing as
Man -- it is a word. There are only men. No! It is not
Man who will be omnipotent, it is some one man, some
immortal man. . .

-- C. S. Lewis, _That Hideous Strength_
(the fictional companion to _The Abolition of Man_)

;->