Bulk skimming this week's AI papers - Sept 20, 2024

Read/listen to the newsletter/podcast on substack:

My Twitter, LinkedIn, Discord, Patreon, consultation booking page, etc:

Timestamps:
00:00:00 intro
00:38:57 Outro

[AISlopForHumans]

Holy shit the time stamp with the sources is unreal subbed for sure

[densonsmith]

"Delves" are a thing in World of Warcraft now so there is a deluge of human written text containing "delve".

[xue]

Really appreciate your work. But I have a question. As you are working alone, how to get enough compute resource for training and testing? As big techs train models with 100 thousand or more GPUs, if you don't get a job in those companies, you would never have the experience with huge GPU clusters. How can individuals compete with big techs in AI research?

[madurarajapakshe]

Nice man !! were you able to find good papers related to malware detection using AI ?

[kakurocksman]

IDK how prompt engineering is going to be antiquated... The new paradigm unfolding around test time compute seems to heavily rely on prompt engineering... You are generating a sampleset of output and sourcing the best candidate "thoughts" from a breadth of generations, how is this not prompt engineering? It's become a lot more than simple Input output prompting, the entire idea of prompt engineering is evolving, NOT being replace with RL.

Not just that, but smaller non SOTA models can utilize ToT prompting to greatly enhance accuracy across a wide range of problems and use cases. This new scaling paradigm can be applied to a wide range of models, and seems to utilize the LLMs inherent randomness as a feature that can enhance the potency of creative programs in the distribution of possible responses to a given prompt.

RL is not going to replace the entire paradigm of Test time compute, atleast for a wile, and considering the multi step reasoning these prompting methods enable, it doesn't seem to be going anywhere for a long time.. Once the next scale of models come, enhanced with the best possible outputs from the previous models, that newer model will than be able to potentially generate improved reasoning with another round of test time compute. IDK how Prompt engineering is going to be antiquated... And even if it does, it's still incredibly useful in order to understand the new developments.

[TheRyulord]

11:40 They actually are referring to Noam Chomsky. It turns out there's a direct correspondence between different kinds of formal languages as defined by Chomsky and different kinds of machines/automata in computer science that can recognize those languages. eg. a finite state machine can model all regular languages but not all context free languages. A pushdown automata can model all regular languages and all context free languages but not all context sensitive language etc.

[HappyMathDad]

People getting better at conveying what they need?? Isn't that the same as prompt engineering???? Why would expressing requirements not be important to learn. I'm sure we're biased, and as humans do not express requirements correctly and accurately.