Panel Discussion

Large Language Models and Transformers

[icriou]

The fact that Ilya is still willing to present and take questions is amazing. Thanks for the organizer and the panel.

[markokedoyin]

Bro after watching Ilya for 50 mins, i swear i'm starting to think ilya is an agi trying so hard to stay on our level and cover his tracks

[bcj]

Ilya: "Pretty dramatic"
Later: "Extremely dramatic"
Me: (grabs popcorn)

[mike_t]

Man, listening to 50 minutes of Ilya dodging questions sure is interesting 🙃

[pcrizz]

I'm wondering if a potential example of exhaustively enumerating over a data set to find a corner cases, might be with the use of an old so called lost language from our history with symbols that have not been deciphered.

I don't know how that would be done, but I would like to imagine that if we seperate what is and is not deciphered and observe if a model attempts to fill in the blanks then I think it would identify exhaustion via extrapolation from other training, assuming it is trained alongside a language with symbology that is considered "complete" or at least not added to.

The plus side is that research into deciphering lost languages is likely under way anyway, and then it may be usable when compared to models that are considered less viable without intervention and/or additional structure.

[yizhe]

Thanks for uploading the panel discussion; there are many interesting questions and discussions here! But I have to say (with all the respect)...this is such a shame and such a bummer to have Ilya here, but he cannot say much for "the obvious reason." I just hope this could be a more open discussion. This is an academic forum, and people come here to discuss and exchange ideas, knowledge, and opinions, supposedly freely. But... here we have "I cannot say much"... sigh.

[voncolborn]

I find it really odd that the panel was not introduced. I know Illya but have no idea who the other people are. They are not even identified on the institutes page for the talk.

[DistortedV]

Barring the error in interpolation vs. extrapolation at 34:00. I could understand Yejin question. She was harkening back to John Schulman’s talk where he said OpenAI would focus on discovering “new knowledge”. Ilya wasn’t biting to provide more insight. A more concrete example is ask if a model could predict the success or lack of success of a scientific idea it wasn’t trained on like LK-99 given the details of the experiment, etc.

[mostafatouny]

Thanks for the joyful discussion. The jokes alone made my day.😊

[odiseezall]

The hard truth is that right now intelligence is an engineering problem, not a scientific one. Which means we will see it work without understanding it. Thus, we have to look closer in the mirror and understand ourselves better first. People are lying to themselves - about consciousness, intelligence, humanity, morality... How do you expect to understand neural networks if you don't even understand yourselves?

[shrek]

how to find this video whe clicking under their channel? its not under videos or live?

[riverland]

That’s my professor in the background getting absolutely Obliterated 🤦🏾‍♂️

[RaviAnnaswamy]

Ok first we say oh it can’t learn this, then it learns it, we say oh it learns with so much data, wait learning to create is valuable first we should acknowledge

[gulllars]

The guy in the pink shirt is just waaay off on his future timelines trajectories. I'm almost surprised how well Ilya is restraining himself in many respects here. Great to see him on the panel, but it being a panel detracted a lot of it for me compared to a 1-to-1 chat with him and someone taking questions there. The other speakers didn't have much to contribute in comparison, an exception being the guy on the far left who had some good comments.

[RaviAnnaswamy]

It is like two empirical scientists sitting in a panel with metaphysicians would prefer to settle via debate but still some good ideas came out.

More like the guy has seen Jupiter’s moons and is arguing heliocentric model but can’t bring his reflecting telescope to others well versed in geocentrism

(I know that was a bit too much but just for fun)

[dotnet]

ilya is just annoyed to death with these other panelists. The inventor of gpt is in the panel, everyone just leave!

[DistortedV]

Ilya - “Absolutely and emphatically not. Absolutely not.” Lol Ilya sounds like a mom scolding child here: 13:45

[jmong]

Don't go on a panel if you’re going to be selfish and secretive about your knowledge. The other panelists seem to be having fun, engaging with and sharing their thoughts and observations while Ilya acts as if he'd rather be somewhere else.

[mrpicky]

someone should explain to all the sub 100 iq interviewers how things actually work in any busyness . that metric of any busyness is one profit/expenses. the rest is secondary. so ChatGPT we get is the cheapest they could make. and if they can make it even somewhat worse but substantially cheaper- they will. even if they already solved everything they would take some time before revealing it or even intentionally bury.

[jasonandrewismail]

DISSAPOINTING PANEL, MAYBE REMOVE COMPUTER SCIENTISTS OF THE TASK SINCE THEY DONT SEEM TO KNOW WHAT THEY ARE DOING