[ML News] AI Threatens Biological Arms Race

#mlnews #gtc22 #ithaca

Your regular updates on what's going on in the ML world!

OUTLINE:
0:00 - Intro
0:20 - Register to Nvidia GTC and win a 3090!
4:15 - DeepMind's Ithaca deciphers Lost Ancient Texts
6:45 - Drug discovery model turns toxic
10:00 - Gary Marcus: Deep Learning is hitting a wall
19:40 - GopherCite: Backing up answers with citations
22:40 - Yoshua Bengio appointed knight of the legion of honour
23:00 - Meta AI tags parody account of Yoshua Bengio
23:40 - Building games using just natural language
24:55 - YOU.com adds writing assistant
25:45 - Horace He: How to brrr
26:35 - Karpathy: Reproducing Yann LeCun's 1989 paper
27:50 - Pig grunt emotion classifier
28:20 - AI annotates protein domain functions
29:40 - Atwood & Carmack: 10k self-driving car bet
30:50 - Helpful Things

References:
Register to GTC and win a 3090!

DeepMind deciphers Lost Ancient Texts

Drug discovery model turns toxic

Gary Marcus: Deep Learning is hitting a wall

GopherCite: Backing up answers with citations

Yoshua Bengio appointed knight of the legion of honour

Meta AI tags parody account

Building games using just natural language

YOU.com adds writing assistant

Horace He: How to brrr

Karpathy: Reproducing Yann LeCun's 1989 paper

Pig grunt emotion classifier

AI annotates protein domain functions

Atwood & Carmack: 10k self-driving car bet

Helpful Things

Links:

If you want to support me, the best thing to do is to share out the content :)

If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n

[KevinHorecka]

I made a model that was fairly effective at predicting whether a dog would survive canine parvovirus or not given their stage in the illness, and I had to opt to not release it because of fear that organizations would use it to decide who to kill, instead of who to provide extra treatment for. So the "every model can be abused" message in this video really resonated with me.

[seetj]

As an organic chemist, synthesis is a very difficult part which takes a lot of trial and error to make it work. Often times we are only able to get very low yields of the compounds that we wanted to make.

[YannicKilcher]

Yes I know my sound is crap in this one, sorry :-S
OUTLINE:
0:00 - Intro
0:20 - Register to Nvidia GTC and win a 3090!
4:15 - DeepMind's Ithaca deciphers Lost Ancient Texts
6:45 - Drug discovery model turns toxic
10:00 - Gary Marcus: Deep Learning is hitting a wall
19:40 - GopherCite: Backing up answers with citations
22:40 - Yoshua Bengio appointed knight of the legion of honour
23:00 - Meta AI tags parody account of Yoshua Bengio
23:40 - Building games using just natural language
24:55 - YOU.com adds writing assistant
25:45 - Horace He: How to brrr
26:35 - Karpathy: Reproducing Yann LeCun's 1989 paper
27:50 - Pig grunt emotion classifier
28:20 - AI annotates protein domain functions
29:40 - Atwood & Carmack: 10k self-driving car bet
30:50 - Helpful Things

[NoNameAtAll]

the 6:45 part is chemical warfare, not biological

[ClaudeCOULOMBE]

For your information, Yoshua Bengio was born in Paris, and his parents immigrated to Montréal, when he was a child. His brother Samy Bengio is also a ML scientist at Apple and an ex-Googler.

[TheEbbemonster]

"Full Self driving will be ready in 2 weeks" - Elon Musk

[steveholmes]

Awesome updates. One thing about symbolic computing, our brains had to create actual silicon calculators and symbolic computers, like wolfram alpha, to make huge steps in science, so natural to think that AI would need to to the same.. vs just bigger neural nets.

[Rhannmah]

6:49 this is not something to be scared of, on the contrary. Having a system that can generate biochemical weapons is extremely beneficial because it democratizes the knowledge of them. The big scary thing about these kinds of weapons is facing unknown chemicals/pathogens that you do not know how to counteract. If all the compositions and countermeasures are already common knowledge, these weapons become way less threatening.

[sammay]

More ML models need to do multiple tasks like the ancient text system. Predicting missing text + origin + time written. I think the future of ai has to do more tasks at once, even mixture of experts is better than nothing.

[paxdriver]

Yan - "Welcome to ML News, it's Monday... "
Me - "Excellent... 😁"

[CosmiaNebula]

coming next:
Biologicals Threaten AI Arms Race

[gabrielb.]

I found your thoughtfulness "if I ask a human the same question I will be happy with that answer" quite dissapointing.
You will be happy because you imply a specific context (casual conversation between two adults), but I dare you to change the statement to "if a child ask me this question", then the thoughtfulness test makes all the sense.

The real value of the thoughtfulness test is to show us how far are we from a black box product, can this model be used by someone who does not known what a language model is?

BTW: Great work! love it!

[Kram]

I think a lot of symbolic reasoning comes down to recognizing, distinguishing, and modifying types. (i.e. recognize that some stimulus is a number or sentence), and I think sparsity is actually a great way to make models go towards typing: In order to remain sparse, they will have to discriminate contexts. I suspect quite a bit of our brains' abilities to do symbolic reasoning is going to be a consequence of its sparsity.

[VladimirZmurikov]

If symbolic thinking is needed to speak and communicate, then some of the rules and patterns are already contained in speech, and in this case the machine learns some symbolic mapping.

[KamilCzerski]

In the next Tomb Raider, Lara Croft should use Ithaca to figure out ancient languages

[Kram]

"Bad Examples" such as that RedBull question are clearly more about AI misalignment than they are about the AI actually giving a bad response.
The AI has no idea about pop culture as a concept to watch out for. It's just presented with huge amounts of data that is necessarily gonna be inundated with pop culture, and it's given no mechanism what so ever to distinguish such situations from "The Actual Real Objective Truth".
Pretty sure to do that you'd either have to somehow painstakingly tease apart dry "pure" reality from any sort of additional associations (an approach that has failed to work for decades for being neither scalable nor well-defined), or you'll have to go more towards embedded AIs with "lived experiences". Some sorts of methods to figure out what's "real" and what's a metaphor.
Like, the signal right now is either hard-coded, limited and brittle, or it is non-existent. Why would anybody even expect that AIs magically can learn "the right thing" that way? AI still isn't magic, even if it can occasionally give that impression.

[serta]

A group of scientists training the biggest multilingual neural network with 176 billion parameters on Huggingface

[scottmiller]

I had no idea there were talking unicorns in Homer.

[herp_derpingson]

8:00 If I wanted toxic things, I would just visit 4chan ;)
.
24:00 From mid 2000s people have been trying to replace programmers with no-code type programs and all of them fail miserably because programming is inherently a complex task. Even if you replace code with drag and drop stuff. It still needs to encode the complexity that the software needs to work. So, the result usually becomes an unreadable mess of boxes and lines which is even more complex than a few hundred lines of python it was supposed to replace. I dont have high hopes for GPT codex.
.
30:00 How do you define a "major city"?. Self driving cars are illegal in India due to the fear of loss of jobs.

[cbongiova]

2030 will not be ready. It will be very close but not quite there.