Prof. Noam Chomsky, the father of modern linguistics, talking about ChatGPT #machinelearning

[IlhamTauf]

Being in this age and can still clearly pour out structured argument. Amazing

[jesselopes]

What people seem not to realize (yet) is that LLMs are a huge confirmation - not disconfirmation - of Chomsky's approach - LLMs need huge amounts of training & data to approximate human language - humans need virtually no training and many orders of magnitude less data - this is called the poverty of the stimulus argument - it's the basis of Chomsky's approach

[jesselopes]

Everyone, y'all need to realize he's making a single philosophical point: LLMs don't satisfy the principle of sufficient reason: 'why something is one way rather than another way' - this is the basic principle of ALL science - so LLMs are not science because for these models "anything goes"

[lndeep]

They demonstrated what scaling up a simple learning system (transformers) with few inductive biases can do in terms of language generation. That was the main claim of the GPT papers and I also believe that they claimed that specific task performance is an emergent property of a broad generative model at scale. So you can have a single model that does it all. I regard it as a simple empirical proof of existence. It’s a contribution, but just not scientifically very interesting. Imagine studying the complexity of language your whole life to discover that a fairly simple mechanism at scale can outperform most other approaches and reduces most future funding on the ground of improved performance using traditional methods out of the window. Traditional methods go from being engineering to humanities and get the same level of funding. The beauty of the phenomenon of language is also unmasked in a very disappointing way. Again… only in terms of engineering solutions. It’s still as beautiful and rich as always. It’s just that new insights won’t substantially build better systems and funding dries up because of that.

[muhammed]

A legendary linguist monologuing on computer science over phonk music? nothing catches my attention better than this edit

[siddhadevapps]

Or, LLMS and ChatGPT in particular have achieved something - they made it obvious, that eloquence is not the same as intelligence. (No intent to bash linguists ....but... you get the point :-))

[martins]

Wasnt there some Model doing actual science, simulating and testing and discovering new molecules?
And yes chatGPT isnt really doing much in terms of science, because it wasnt made for it. But it's still a really impressive tool.

[evanfreethy]

I can guarantee that if GPT 4 was around when Chomsky was coming up and cutting his teeth in the academic world, he would have a completely different interpretation of where our future is headed due to the existence of these models

[abdelrhmandameen]

I love Noam, and I seriously think he is very underrated but he is looking at the whole language models from a very narrow perspective. Soon enough they will be able to drive answers from all the materials that we have and come up with different things and pose totally different questions.
It will change every thing we know about linguists and how we perceived our reality.

[edeneden]

This lacks context, I dont understand what his point is

[CristianGarcia]

Saying this about GPT sounds cool because chatbots are not very scientific, but what about alpha fold?

generation > understanding

[tripptrump]

Can we all evaluate what we are we doing determining how humans can stop evolution? That's when CHAT WILL conciede .

[zynot]

Complete cope. His life's work made useless.

[christopherhitchens]

This is fake, it’s not even Noam speaking

[alexkang]

Lol i love noam he is on with sigma music

[tostupidforname]

What is he talking about? His point is that gpt-3 is just a larger model? Sure but that shows us what can and can't be done when we increase the amount of training data and im sure this was not obvious before.