AlphaCode Explained: AI Code Generation

AlphaCode is DeepMind's new massive language model for generating code. It is similar to OpenAI Codex, except for in the paper they provide a bit more analysis. The field of NLP within AI and ML has exploded get a lot more papers all the time. Hopefully this video can help you understand how AlphaCode works and what some of the key takeaways are.

[huanvo]

One of the better explanation of AlphaCode, I'm glad I stumbled upon your channel. It would be even better if you add timestamps to the video so then the viewers can jump to the section that they are interested in

[Azns]

I just love How fast Technology is changing.

[singhaladitya]

Fantastic video Edan. Liked and subscribed. And really grateful.

[jizhang]

Thanks for sharing your thoughts at 2am before a flight.

[hobrin]

great vid. I don't actually like reading papers but I like hearing papers, if you get what I mean. And with somebody stating their opinion on it and saying like: yeah this part is not interesting. That's is very nice to hear.

[viniciusleitepereira]

Great video, man! I've just hit that subscribe button!

[drdca]

I think it might be interesting to see a model which is trained on like, git commits, where, analogous to translation tasks, it is given as the context, the file before changes are made, and the commit message, and what it is to output is the version of the file after the commit .
Though, I wonder if one pitfall for that might be that if first learns that most of the file is the same, and so it just learns to copy the file, and any time it starts to learn any changes to the file, it makes the chance that it adds changes where it shouldn’t do so too large, resulting in unlearning it?
So that it never learns to make the changes?
If that issue happens, maybe it could be resolved by having a separate part of the model which predicts what parts do and don’t have changes, or where the first change is, or similar, and then use the part that predicts what the code in the changed version should be, based on that?

[rednationsox]

How can transformer handle such long sequence (problem statement + code) in training?

[dl]

1. Does AlphaCode test it's generated programs (compile, execute, and parse results)? From experience that's the most successful iteration loop I perform during development.
2. Does AlphaCode leverage GPT-3? I didn't hear it mentioned (maybe I missed it). If not, would that help?
3. What order of magnitude of processing power does this use? ie. does this consume $1 or $1000 per problem in energy and reserved compute time?

[stephenkamenar]

50 percentile is insane. being in the 1 percentile, being better than only like 1 human in the competition would still be impressive.
AI couldn't write any code just a few years ago

[kiralight]

It’s so scary to see AI developing so fast that it is on track to replace even jobs like programmers. This was only the first half of 2022, can’t imagine what it would become in 2030.

[mully]

To someone only just becoming aware of deepmind alphacode, is this basically a AI that writes the code you explain it to without doing the code yourself ?

[mbappekawani]

are you correct for a question if your answer is within the top 10 hypotheses? and the point is that they can all be tried and one of them will be the right answer? If so then its really important to emphasize that this is not accuracy in the sense that you get one attempt

[Fircasice]

The reason human programmers need the iterative process is because they can't just "see" the solution to a problem. I think the ultimate goal here is to enable an AI to be presented with a problem, fully understand it and "see" a solution at first glance, not create something that has to follow a lengthy process like humans do.

[johnstifter]

When you say you're critical of its scaling do you mean you're critical of its ability to take one task and generalize from that to solve various similar tasks, kind of like transfer learning?

[hatsu]

I need this a.i to help me understand the process flow of a function and methods pass through the memory of a computer, OS instructions.

[shashankroni]

This doesn’t make sense to me, given the No Free Lunch theorem.

Deep learning doesn’t work in an absolute sense; it works relative to certain assumptions, with certain architectures, and certain training regimes; it also frequently fails (though not all failures are published).

[alexandrsoldiernetizen]

Almost like a genetic programming methodology rather than machine learning.

[zackcarl]

After watching this I think i as a computer programmer and development don't matter he can do it better

[qwertyqart]

Depending how long it take for It to become average, let’s see where is going to be in 6 months.