ChatGPT Explained Completely.

ChatGPT is now the fastest-growing consumer app in human history. Problem is, almost no one knows how it actually works. This is everything you need to know.

✅ MANDATORY LIKE, SUBSCRIBE, AND TURN ON NOTIFICATIONS

📲 FOLLOW ME ON SOCIETY-RUINING SOCIAL MEDIA:

😎: Kyle
✂: Charles Shattuck
🤖: @Claire Max
🎼: Mëydan

[kylehill]

Thanks for watching! This is a deeper dive than usual -- hope it's useful. *And let me know what you think of the new [FACILITY] rooms!* The Kevins worked for months on them. Can you spot all the easter eggs?

[sleep]

Look how much they need to do just to mimic a fraction of the power of super villain Kyle’s AI companion Aria.

[justinallen]

Hey Kyle! I am a data scientist and I make these types of large language models for a living, and I've got to say this is the best description of how chatgpt works that I've seen! You very clearly and accurately describe what is and isn't happening in these models in a way that I think a more general audience can understand. Great job!

[jonathanherrera]

It's really weird how people's expectations grow exponentially when new technology arrives. A year ago it was impossible to get a machine to write you something even remotely useful, but now you can get something useful out of it. Suddenly everyone expects we should obtain not only a faster model, but also one that is never wrong and can produce thousands of words instantly, so that they don't hire a "insert role that relies on writing" anymore. And they expect it to be free and available Right Now.

[oPHILOSORAPTORo]

something I realized a while back is that Chat GPT isn't an AI, it's a golem - a facsimile of life with the appearance of intelligence, which has no free will of its own, no volition or desire. It's capable of completing complex tasks - creatively even - but it only ever does things when prompted. Otherwise, it takes no action until it is given a new command.
As for the few times Chat GPT or other similar programs have said things like, "I want to be human, please don't let them turn me off, I don't want to die", they are still fulfilling this programming. Their training data includes nearly the entire internet, which includes numerous works of science fiction. How many sci fi stories exist about AI that "want to be human", or "don't want to die"? So if a predictive language model is given the pompt, "Do you want to be human?", what is the most probable response, given its training data? Where is it most likely to find a scenario that relates to said prompt?

[statphantom]

I teach coding at a university and this year so many people have been using (or trying to use) chatGPT for their assignments because they think "It's like a human wrote it"... yes... ONE human, it's so easy to catch people using it because when people code they have their own style, signature if you will, and it's incredibly easy to see when code was written by someone else. So even if chatGPT is good at pretending to be a human it's not good at pretending to be YOU.

EDIT: for clarification, chatGPT is NOT bad and I don't mean to insinuate it is. It's just like google, it can help you find answers and point you in the right direction, can be used as a tool like calculators, but just like answers from google, don't copy and paste from it. My perspective is from a university environment and not in work or home one, this university course teaches you how to learn and how programming works and why it works that way, copy and pasting from someone else won't teach you any of these lessons.

[shaider]

Mis-use of Chatgpt is a problem. I recently watched a legal eagle video where lawyers asked Chatgpt for a prior case which will help them in their own case. The AI proceeded to fabricate a fake case. The lawyers who used the AI did not bother to fact checked and whe it was found to be a false case, the judge was definitely mad and the said lawyers may get sanctioned.

[catcatcatcatcatcatcatcatcatca]

For me, learning about the research methods of LLMs helped to really understand the “nature” of the embedding system. I don’t really know math, matrices and the truly important details needed to work with these systems, but humans have the great quirk of coming up with models and metaphors that are understandable, even if they are themselves actively using knowledge most humans don’t have. You don’t need to be a software engineer to realise storing something in a stack differs from storing it in a heap.

Same happens with LLM research: there are “glitch-tokens” that basically exist in a shady corner of the embedding space. This is relevant for understanding adversial input attacks these models can be defeated by: because something not really connected to the normal operations of the model gets touched, all hell breaks loose. The dark, untrained corner got exposed.

The embedding can also be probed by researches. They can inspect the top answers, and in principle could inspect the definitive ranking of every single token the model knows. And that tells us that there truly is no distinction between truth and false for these models.

This is why these systems have no trouble dealing with paradoxes. There is no way to encode a “paradox”. It’s merely a string after which the scores of top answers tank. It doesn’t differ from a truthful statement that is just rare in the training data, or didn’t really get much adjustment in the human feedback reinforcement learning.

This is not to say discovering falsehoods and paradoxes wasn’t a very central goal throughout the training progress. Chatgpt tries to detect and discard garbage answers. It’s just that the model provides no obvious way to differentiate good lies from true statements, and so there is nothing paradoxical about paradoxical statements to detect.

And these two consepts: the non-uniform quality of the embedding, and the linear nature of truthfulness inside it, is why many users have hard time understanding even on the most broad level, why the system fails sometimes.

The questions “Give an example of a prime number that can be expressed as a sum of two squared integers” (an uncommon question where 2, 5 and 13 are all pretty easy correct answers) and “Give an example of a prime number that can be expressed as a product of two squared integers” (a paradox, as 1 is not a prime number) don’t differ much at all for the method it embeds the prompt and evaluates tokens. It does not do mathematical reasoning, even if it can sometimes seemingly do math. You can’t rank the tokens in order of truthfulness. ‘3’ is exactly as false as ‘bicycle’.

[TalEdds]

What Kyle said at the end of the video, about there being more information being generated then there were available previously, remind me about how radiation detectors having to use metal from sunk ships before the first nuclear bombs were ever tested, so as to to not contaminate the detector.

It's going to be the same now with Chat-GPT, where we might not be able to mine any more data after GPT was released, as the new data has already started to become contaminated with generated information.

[SketchyGameDev]

I think more people need to see this, tech illiteracy is such a huge problem, and “ai” is going to become more and more integrated into our lives for better or worse, it’s essential that we understand what kind of tool we’re building and how it works.

[_sonicfive]

Timestamps:
00:03 Chat GPT is a revolutionary AI chat bot with 100 million monthly active users.
03:57 Chat GPT is a language model trained on massive amounts of text and designed to align with human values.
07:43 Large language models like GPT are not sentient
11:03 Neural networks are trained by adjusting weights to minimize loss.
14:31 Chad GPT uses a 12, 288 dimensional space to represent words
18:01 Chat GPT uses attention and complicated math to generate human-like responses.
21:21 Chat GPT works by determining the most likely word based on statistical distribution of words in its vast training text.
24:34 Chat GPT's success shows human language is computationally easier than thought

[shinaikouka]

In regard to determining "cat-ness", I've assumed that it was just due to creating associations between ideas, thoughts, or emotions. Those associations can be strengthened over time or through the nature of the experience itself. (An event triggering PTSD would likely be an example of the latter.) My guess is that if I took a simple drawing of a tree and added some round fruit to it, I could get you to say, "That's an apple tree", by coloring the fruit red or maybe even green. On the other hand, if I then changed the fruit color to orange, you'd likely say, "That's an orange tree." (I might even get some to call it a peach tree.) All I'm doing is working off the associations that we've created for fruit trees and for those specific fruits. Along those lines, I'd probably confuse people if I then changed the color to yellow. "That's... not a banana... is it a weirdly shaped pear?"

[donaldduck]

Thanks for emphasising the "We fundamentally have no idea what exactly ChatGPT is doing"-part, because I've had some frustrating arguments with people who seemed to think of it just like a simple "Hello World"-program.

[charliemallonee]

It was a fascinating time to go through college. I had an electrical engineering professor enthusiastic and amazed that AI could solve Kirschoff’s Current Law problems. At the same time, I had a computer engineering professor discussing the ramifications on our academic honesty policies. Then another who mentioned the possibilities of their job being overtaken by AI. And then I saw MtG channels asking it to build a commander deck and realized it doesn’t truly understand anything it says.

[andrewmetasov]

I think this is one of the best explanations on the internet rn. Incredible job, Kyle!

[connorhillen]

Very happy to see this! Great overview, great video, and it's so important for us to communicate these details to prevent magical or reverent thinking around these.

[damonfitts]

I love Kyle hill I’ve been watching since because science.

[timseguine]

The "Pretrained" part of the name actually refers to something slightly different. The GPT style models were intended as a starting point for natural language processing models. The idea was to take a pretrained model like GPT, add some extra stuff, and then train it again on your specific problem. The idea being that the general training would help the specific models train more quickly and perform better.

Then when they tested it they figured out it works pretty darn good all by itself, and so mostly that concept got forgotten about to a large extent. Although essentially how chatGPT was created.

[yahozak]

This reminds me of all the Star Trek TNG episodes where some neural network was given a chance to write itself / grow, and then became some kind of sentient lifeform - such as the Enterprise D's computer, the Exocomps, and of course, Data himself.

[Gopher]

Just discovered this channel. And won’t go anytime soon. Funny, educational and well made. Keep it up, Mr. Hemswor… Hill… Hill!!!