Transformer LLMs are Turing Complete after all !?

preview_player
Показать описание
Are transformer LLMs equivalent to Turing machines or not? Spoiler: they are, because Franz Nowak (the guest of this episode) and colleagues proved this in their latest paper!
We talk with Franz about RNNs, transformer encoders, decoders (with CoT), so if you're like me and always wanted some clarity on the computational expressivity of deep learning architectures -- and the Chomsky hierarchy -- have a listen! 🎙️

Outline:
00:00 Transformers are Turing complete!
00:52 Franz Nowak Intro
03:14 The Chomsky Hierarchy
07:49 Is my laptop Turing complete?
09:27 Transformer encoders
10:19 Transformer decoders
12:26 RNNs are Turing complete
13:52 LLM with CoT
17:51 Learnability
21:11 Sparsity of human proofs
25:59 Why care about theory?

Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏
Dres. Trost GbR, Siltax, Vignesh Valliappan, Michael, Sunny Dhiana, Andy Ma

▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕
Join this channel to get access to perks:
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

🔗 Links:

#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research​

Music 🎵 : Just Breathing (Instrumental) - NEFFEX
  1. AI Coffee Break with Letitia
  2. On the Representational Capacity of Neural Language Models with Chain-of-Thought Reasoning
  3. transformers are universal turing machines
  4. LLMs are turing machines
  5. LLMs with CoT are universal
  6. computational expressivity of transformers
Рекомендации по теме
Transformer LLMs are 0:28:47

Transformer LLMs are Turing Complete after all !?

Transformers (how LLMs 0:27:14

Transformers (how LLMs work) explained visually | DL5

What are Transformers 0:05:50

What are Transformers (Machine Learning Model)?

Transformers: The best 0:08:38

Transformers: The best idea in AI | Andrej Karpathy and Lex Fridman

How Large Language 0:05:34

How Large Language Models Work

LLM Explained | 0:04:17

LLM Explained | What is LLM

Transformer Neural Networks, 0:36:15

Transformer Neural Networks, ChatGPT's foundation, Clearly Explained!!!

What is Retrieval-Augmented 0:06:36

What is Retrieval-Augmented Generation (RAG)?

What is generative 0:46:02

What is generative AI and how does it work? – The Turing Lectures with Mirella Lapata

How Large Language 0:20:38

How Large Language Models (LLM) In Generative AI Are Trained ?

Reinforcement Learning through 0:10:17

Reinforcement Learning through Human Feedback - EXPLAINED! | RLHF

Transformers for beginners 0:19:59

Transformers for beginners | What are they and how do they work

The Transformer neural 0:10:08

The Transformer neural network architecture EXPLAINED. “Attention is all you need”

LLMs are Culturally 1:02:39

LLMs are Culturally Programmed Turing Machines

Exploring efficient alternatives 0:44:56

Exploring efficient alternatives to Transformer models

PyTorch in 100 0:02:43

PyTorch in 100 Seconds

What are Large 0:05:30

What are Large Language Models (LLMs)?

Why Large Language 0:09:38

Why Large Language Models Hallucinate

What are Large 0:08:16

What are Large Language Models (LLMs) & Transformer Networks: 'autocomplete perfected'

RAG Explained 0:08:03

RAG Explained

Introduction to large 0:15:46

Introduction to large language models

How ChatGPT Works 0:07:54

How ChatGPT Works Technically | ChatGPT Architecture

The Narrated Transformer 0:29:30

The Narrated Transformer Language Model

🚀💬 NVIDIA’s Megatron-Turing 0:00:46

🚀💬 NVIDIA’s Megatron-Turing NLG

Комментарии
Автор

This was super nice! I hope AI Coffee Break podcasts become a regular thing!

DerPylz
Автор

Wonderful podcast, perfect duration, great questions, keep them coming!

hiauoe
Автор

I should read the paper first, then watch this amazing talk again. I did not understand completely. But thank you for this precious content.

marzi
Автор

Great content and format. This channel just keeps getting better.

CodexPermutatio
Автор

Great to see you, I remember you from that beautiful video you made couple of years back on the "attention is all you need" paper

shadysaeed
Автор

Being Turing complete is nice and all, but this does not inherently mean that there will be existing solutions to certain problems inside reasonable computational limitations.

gergelyszabo
Автор

Can His tutorial on computational expressivity of Language Model be found on youtube?

ZafyArimbola
Автор

21:08 Great! Maybe we need an AI so smart that we can ask "solve the problem but make sure you do it in a way that our puny human brains can understand". Weird world we're creating.

rockapedra
Автор

its funny, in the universal transfromer paper they show turing completeness by ignoring the decoder while focusing on the adaptive compute time layerwise instead of context-step-wise "chain of thought" .. so transformers are double-turing-complete? :D

thomasmuller
Автор

Didn't watch yet - at work xD. Did they solve the infinite input width problem?

andresot
Автор

A ticket tape machine is Turing Complete. It's a very low bar

haniamritdas
Автор

llms can generate coherent sentences, they are probabilistic in nature, and based on randomness. randomness/random sampling of the latent space creates halucinations, because there is no reflection, no cognition of any kind hapening. a very advanced automaton, that can imitate humlan language, nothing else, pure imitation. just like a plane isnt a bird, it flies, it goes superfast, but its not a bird. similarly a language model imitates words and sentences, but there is nothing else going on.

genkidama
Автор

Lip stick, east european accent, and AI.

Paplu-it
Автор

They are beyond completeness as their attention mechanism esentially gives them a self destruction switch that flips the context and frame of a problem as it relative hardness requires it.

New Oracle Turing Machines could arise from the attention mechanism. Transformers should definitely be studied further by CS theorists.

TheHouseOfBards