DeepMind's New AI: As Smart As An Engineer... Kind Of! 🤯

📝 The paper "Competition-Level Code Generation with AlphaCode" is available here:

❤️ Watch these videos in early access on our Patreon page or join us here on YouTube:

🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Angelos Evripiotis, Benji Rabhan, Bryan Learn, Christian Ahlin, Eric Martel, Gordon Child, Ivo Galic, Jace O'Brien, Javier Bustamante, John Le, Jonas, Jonathan, Kenneth Davis, Klaus Busse, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Michael Albrecht, Michael Tedder, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Rajarshi Nigam, Ramsey Elbasheer, Steef, Taras Bobrovytsky, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi.

Károly Zsolnai-Fehér's links:

#deepmind #alphacode

[IzUrBoiKK]

Love to see a new paper! Pls keep it up always guys, the team is doing a great job!

[Chemson]

"Make it work or not, either way we'll lose our job." - DeepMind engineers

[ThioJoe]

Oh man, this is getting good. How long until AI can write more AI’s?

[juliandarley]

this definitely feels like a significant step forward even from just a few weeks ago. on the downside, one can ask: how long before it starts to have an impact on coding jobs? but on the upside, it could help many projects which are trying to help humans and the planet, but are not likely to be profitable and therefore currently don't usually get done.

[kdjshfihekls]

It didn't "invent" DFS. DFS is a very common algorithm, so it is likely present many places in the training data. Also if it didn't know dfs by memory how would it know that it's called DFS?

[nidavis]

The biggest revelation from this paper is that most people in the comments have no idea what a software engineer does on a day-to-day basis. Certainly it has very little to do with solving competitive coding problems. In my experience it comes down to understanding human intent much more than understanding a poorly-written requirement, while understanding the vast context of all the other parts of the system around you. What a user asks for quite often does not map to what their intent is -- or they try to describe a solution only based on other irrelevant things they've seen. The hard job is getting to the intent, understanding how it fits into a massively interconnected system, and knowing how and where to implement it while keeping it all maintainable. The amount of time spent designing limited-context algorithms with precisely-defined requirements approaches zero.

[orbital]

AI is 80% hype 20% actual results. No wait, let me fix that. AI is 99% hype and 1% results. This quite clearly falls in the hype category (and its not certainly not the first DeepMind paper of this type). As a computer scientist, I'm so tired of all this non-sensical AI hype by people who don't really understand the limitations of these techniques.

This AI just generates a shitload of programs based on an extremely precise formulation of a simple problem by gluing together code based on samples from humans. Then it filters the programs based on tests. 99% of the programs generated don't even pass the 5 example tests in the problem description, which already tells you that the AI has extremely limited understanding of the code or the problem. It's neat but not nearly as impressive as it sounds. This does not scale to (a) underspecified programs (b) novel programs that aren't just a variation of well-known contest problems with solutions on github (c) anything complex enough to need more than 100 lines or so of Python (d) anything that ever needs to be read or modified by a human (e) anything that you actually want to be correct in general, not just for some limited test cases, etc., etc. Basically, this is about as close to an AI programming by itself as a paper airplane is to a spaceship.

4:40 Give me a break. This is complete and utter nonsense. This AI did not invent DFS or do you seriously think that the AI conveniently invented the name "dfs", too? A simple DFS snippet like that obviously appears in the training data. Something like github copilot can easily "invent" DFS in the same way. The comments on the alphacode website which you cut out in the video, say:

> Here it most likely borrowed large chunks of code from other dfs-on-tree problems, such as the "dfs" function or the I/O. The sorting of adjacent vertices by depth seems to be unnecessary, fitting the pattern of useless-but-harmless code pieces that we see in other AlphaCode submissions.

So the authors make it very clear, that even in this cherry-picked best case, the solution was generated by gluing together code fragments from the training dataset. I find this video to be quite disingenuous. You are *massively* overhyping this paper.

[jonmichaelgalindo]

"It sometimes forgets about unused variables--very human-like."
When AGI shoots way past human intelligence, will it still be stupid enough to doom itself, but on a galactic level? 😨

[tlatitude]

I wonder when the AI's will be able to insert some missing bits of code in some function. As opposed to writing everything linearly like we saw here.

[JoeTaber]

Programmers often use source control software that shows the difference between the previous version and the proposed changes. These are combined in a "Pull Request" which is the unit of evaluation and code review. I'll start paying attention when it can take an existing codebase and a description of changes to make and outputs a diff for review.

[syntaxed]

Need to put these A.I's to work and study propulsion technologies - Perhaps they can come up with some kind of tech concept that lets us reach other planets in a somewhat timely manner.

[thatchessguy]

Darn, I just finished a Compsci and math degree because I thought I would have a guaranteed job. Guess I was just replaced, or at least in the next 10 years.

[FreekHoekstra]

I would love to see how scalable and generalizeable it’s solution is compared to the average human programmer. And can it work in an established codebase?
Success rate is amazing but it’s really the quality of the code that matters in the end, still this is an absolutely amazing step.
Soon we can talk to our computers in general English and we won’t need to worry about syntax. that sounds like a glorious day to me, where I can just focus on the algorithm instead of the syntactical oddities of each language.

Regarding DFS the fact that it actually named DFS means it probably learned it rather than invented it, still totally amazing

[cancerino]

It's also clear it thinks very differently from humans. It solves the problem as a text writting problem, not an algorithmic one, and "character by character", simply computing the probability of which character should be inputed next.

[aryasenadewanusa]

now I'm convinced that you're no longer using your own voice, it sounds like a very good AI generated.

[SmartK]

Every programmer knows that programming is 90% Google, 9% laying around thinking about the problem, 1% boredom of actually writing it. Of course programming is just a tiny sliver of actual work, that consists mostly of debugging, testing, cursing and thinking about building a log cabin in the middle of the woods, where you'll live without electricity for the rest of your life.

[julinaut]

my brain goes to weird places whenever i see an ai output blend in with human output in a dataset. that is turing test level goodness. What a time to be alive indeed

[stanislavspacek]

This is absolutely huge. This means that pretty much any idea can be turned into reality with just a text prompt in just a few years. Combine that with the fact that were slowly being able to understand our own thoughts. Meaning we could literally think of something and show a render of it to someone else. That's just one of the impossibly huge amounts of applications. Maybe giving it a program and just telling it to optinise it. Maybe even change it to be more practical and ergonomic. As you said - What a time to be alive!

[garronfish]

This channel started showing interesting stuff now things are starting to get scary. However as a programmer I'm not nervious yet as I know how little gets into the next release of software compared to the number of possible improvements stuff like so this will really speed up the process. I just hope it leaves some fun stuff for us humans to do.

[vs]

Very impressive! But in a sense wouldn’t writing out all the instructions for an algorithm be just the same as coding it.