Joscha Bach - Why Your Thoughts Aren't Yours.

Dr. Joscha Bach discusses advanced AI, consciousness, and cognitive modeling. He presents consciousness as a virtual property emerging from self-organizing software patterns, challenging panpsychism and materialism. Bach introduces "Cyberanima," reinterpreting animism through information processing, viewing spirits as self-organizing software agents.
He addresses limitations of current large language models and advocates for smaller, more efficient AI models capable of reasoning from first principles. Bach describes his work with Liquid AI on novel neural network architectures for improved expressiveness and efficiency.
The interview covers AI's societal implications, including regulation challenges and impact on innovation. Bach argues for balancing oversight with technological progress, warning against overly restrictive regulations.
Throughout, Bach frames consciousness, intelligence, and agency as emergent properties of complex information processing systems, proposing a computational framework for cognitive phenomena and reality.

SPONSOR MESSAGE:
DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)?

TOC
[00:00:00] 1.1 Consciousness and Intelligence in AI Development
[00:07:44] 1.2 Agency, Intelligence, and Their Relationship to Physical Reality
[00:13:36] 1.3 Virtual Patterns and Causal Structures in Consciousness
[00:25:49] 1.4 Reinterpreting Concepts of God and Animism in Information Processing Terms
[00:32:50] 1.5 Animism and Evolution as Competition Between Software Agents

2. Self-Organizing Systems and Cognitive Models in AI
[00:37:59] 2.1 Consciousness as self-organizing software
[00:45:49] 2.2 Critique of panpsychism and alternative views on consciousness
[00:50:48] 2.3 Emergence of consciousness in complex systems
[00:52:50] 2.4 Neuronal motivation and the origins of consciousness
[00:56:47] 2.5 Coherence and Self-Organization in AI Systems

3. Advanced AI Architectures and Cognitive Processes
[00:57:50] 3.1 Second-Order Software and Complex Mental Processes
[01:01:05] 3.2 Collective Agency and Shared Values in AI
[01:05:40] 3.3 Limitations of Current AI Agents and LLMs
[01:06:40] 3.4 Liquid AI and Novel Neural Network Architectures
[01:10:06] 3.5 AI Model Efficiency and Future Directions
[01:19:00] 3.6 LLM Limitations and Internal State Representation

4. AI Regulation and Societal Impact
[01:31:23] 4.1 AI Regulation and Societal Impact
[01:49:50] 4.2 Open-Source AI and Industry Challenges

Refs (URLs in shownotes):

Chalmers, D. J. (1995). Facing Up to the Problem of Consciousness. Journal of Consciousness Studies, 2(3), 200-219. [0:52:35]

Chollet, F. (2019). On the Measure of Intelligence. arXiv:1911.01547. [0:07:45, 0:08:30]

Chollet, F. (n.d.). ARC-AGI. GitHub. [1:25:00]

Chomsky, N., & Herman, E. S. (1988). Manufacturing Consent: The Political Economy of the Mass Media. [1:31:40]

Christiansen, M. H., & Chater, N. (2022). The Language Game: How Improvisation Created Language and Changed the World. [0:33:05]

Clark, A., & Chalmers, D. (1998). The Extended Mind. Analysis, 58(1), 7-19. [0:33:20]

Clay Mathematics Institute. (n.d.). The Riemann Hypothesis. [0:07:00]

Commodore 64 Wiki. (n.d.). BASIC. [0:19:50]

Dawkins, R. (2006). The Selfish Gene: 30th Anniversary Edition. [0:16:42]

Descartes, R. (1637). Cogito, ergo sum. In Discourse on the Method. [0:45:35]

Goff, P. (n.d.). Academic Papers. [0:50:55, 0:58:30]

Goodfellow, I. J., et al. (2014). Generative Adversarial Networks. arXiv:1406.2661. [0:24:00]

Hasani, R., et al. (2020). Liquid Time-constant Networks. arXiv:2006.04439. [1:06:40, 1:10:25]

House of Lords. (2024). Large language models and generative AI. UK Parliament. [1:40:15]

Lovelock, J. (2019). Gaia theory. Nature, 570, 441-442. [1:00:30]

Marcus, G. (2024). Taming Silicon Valley: How We Can Ensure That AI Works for Us. [1:26:07]

Maturana, H. R., & Varela, F. J. (1980). Autopoiesis and Cognition: The Realization of the Living. [0:31:25]

Mordvintsev, A., et al. (2020). Growing Neural Cellular Automata. Distill, 5(2), e23. [1:19:00]

Stability AI. (n.d.). Stable Diffusion. GitHub. [1:50:25]

Stanford Encyclopedia of Philosophy. (n.d.). Animism. [0:30:35]

Stanford Encyclopedia of Philosophy. (n.d.). Embodied Cognition. [0:33:20]

Stephan, A. (2006). The dual role of 'emergence' in the philosophy of mind and in cognitive science. Synthese, 151(3), 485-498. [0:17:10]

Tononi, G. (2004). An information integration theory of consciousness. BMC Neuroscience, 5(42). [0:44:55]

Wolfram, S. (2020, April 14). Finally We May Have a Path to the Fundamental Theory of Physics... and It's Beautiful. [0:45:55]

Zuckerberg, M. (2014). Move fast and break things. Harvard Business Review. [1:35:15]

Shownotes:

[MachineLearningStreetTalk]

DO YOU WANT WORK ON ARC with the MindsAI team (current ARC winners)?
MLST is sponsored by Tufa Labs:
Focus: ARC, LLMs, test-time-compute, active inference, system2 reasoning, and more.
Future plans: Expanding to complex environments like Warcraft 2 and Starcraft 2.

[Walter]

I don't feel bad that my girlfriend left me now. There's a new Joscha Bach video!

[stewartcohen-jones]

This man is hugely impressive. Still relatively young. Keep a very keen eye on his progress. A genuine public intellectual.

[jhau]

One of if not the most important scientists of the modern day. A man who not only knows his stuff but knows how to effectively communicate his knowledge, especially in such a way that is profound and awe-inspiring. Thanks for hosting Joscha again.

[vadimcoj]

This is the type of person who you can't listen at 2x, only 1x with pauses

[ArmirCeliku]

My man, you have been on fire with both content quality and quantity lately.

[Terminator-xx]

*_OMG, this man blows my mind every time I set through one of his talks man. I don't really know how to describe someone that's more genius than genius. I know the model when I see it though, and it's this man. It's 100℅ Joscha!_*

[FRandAI]

I think this is the best interview with Joscha Bach yet, which was a tall order.

[renereiche]

I know he has a lot of repeating topics and ideas he always talks about, but not only are they brilliant and very much worth be emphasized and repeated, but whenever you get Joscha off the course of his current topics of interests, you still just get these tremendous(!) often-times spontaneous feeling incredibly insightful opinions and ideas. Just brilliant to listen to this man.

[AICoffeeBreak]

Another Josha Bach appearance, yippi!!🎉

[JoePUNK_Was_Here]

Joscha is by far the most honest interlocutor when it comes to his humble approach to philosophy and the juxtaposition to honest scientific epistemology. His credibility shines bright

[dru]

Whenever Joscha veers off into a different topic, let's say - Christianity, he always somehow provides the best inisght about the topic that I have heard to-date, giving the most eloquent and coherent response, Almost as if he'd been thinking about it for decades. It's honestly mind-blowing.

[csbarolaca]

2 hours of JOSCHA BACH paodacst is equal to 100 hours of any other person podcast:

The WISDOM RAINS LIKE ANYTHING, its not knowledge but the knowledge distilled into wisdom.

After a time people becomes repetetive and you know what they will say next and you Transcedent them.

But with Joscha Bach, I have been hearing him for some 5 years and not being able to TRANSCEDENT him. ONE OF THE GREATES MIND HAVING WISDOM.

HIS THOUGHTS ARE SO COHERENT AND SO DEEPFUL INSIGHTS.

[kaushaltimilsina]

After having studied a bit of Algebraic Topology, Algebraic Geometry and Quantum Field Theory, I am of the perspective; like Joscha Bach that our observation and model of the world is algebraic; that we use generators and relations. I think, where geometry arises is when we try to define a comparison or distance between 2 generators- a distance characterized by some numbers; some kind of a linear form on a pair of generators to another ring/algebra where it is represented.

[MoeShlomo]

I see "Joscha Bach" in the title, I click! Looking forward to this...

[entwine]

Joscha needs to write a book with these ideas. I'd be ready to pre-order as soon as he says that he'll write a book.

[madrooky]

It is fascinating, Josha is one of the few people I can fully agree with on pretty much any topic.

[krollo]

I've always had high admiration for joscha Bach's mind. His thoughts flow so smoothly

[clarkd]

Tim, your channel is my best source for everything AI. Keep up the good work, your channel is invaluable. Thank you.

PS I love listening to Joscha Bach.

[Zirrad]

Uber wins by offloading costs to the drivers, paying them less, providing no benefits, then charging just enough to force competitors out of business, then raising the price back up to what they know the market will bear, the pocketing the difference just for making an app.
Sure the medallion system needed an overhaul, but not sure that Uber is the terrific example he thinks it is.

Otherwise, this was a terrific and thought provoking interview.
Loved it.