DALL-E, Stable Diffusion, Midjourney - AI Image Generation Tested- Revolutionary for Game Devs?

Once a joke producing horrifying abstract images, AI tools like DALL-E, Stable Diffusion and Midjourney are capable of producing some tremendously impressive art. The potential use-cases are numerous - but with art creation being one of the most expensive elements in modern games development, could these tools aid studios in everything from game remasters to concept art to textures? Oliver Mackenzie puts all three of the leading AI image generation tools through their paces.

[HueyTheDoctor]

Disappointed that you didn't generate "a giant enemy crab with obvious glowing weak point you can hit for massive damage."

[victoryeral]

This reminds me of an old star trek TNG episode where Data starts painting to explores his creativity as an AI.
The paintings of his dreams 25 years ago look so similar to what these AI now produce

[phrozac]

Pretty awesome tech and absolutely frightening for digital game dev artists/illustrators like myself. Even though these programs can be used to speed up our work, making us far more efficient by providing a substantial base upon which to work from, it also cuts down on the need for teams of concept artists and texture artists. Whereas before you might need a dozen artists, A.I. art could reduce that number down to 1 or 2. For example, an art director may be all that's needed to generate concept art replacing the need for a concept art team. That's if requirements remain at the current level. I can see this being a boon going forward however as projects continue to scale upwards in scope such as with massive open-worlds or for indie devs with very limited resources being able to produce a large amount of high quality concept art that would normally be out of reach.

[HArryvajonas]

Speaking as someone who went to Art college for illustration and considered going into concept artwork, I did not see this coming. I was an early adopter of Digital illustration (photoshop painting with a wacom) and I have followed A.I. generated artwork over the last decade, The amount it has increased in natural and impressionistic techniques over the last several years is scary. Even though I have followed A.I. art development, I didn't think it could reach the point of mimicking human techniques and applying them in a coherent way and creating original artwork; I thought it would always have a hint of being artificial. I have also followed overall A.I. development across multiple disciplines, and if there was any doubt it will replace nearly all of humanity's tasks and jobs, that notion is completely gone at this point. These A.I. advancements are impressive and equally unsettling.

[RADIOSUICIDIO]

I'm not interested in the AI upscaling of texture assets as much as the possibility of generating endless procedural variations of a given texture, and getting rid of that awkward repetitive "tiled" effect that still plagues most long and flat surfaces

[rstar]

You should also look into NERFs and instant NGP in particular! This technology could already be used for backgrounds in games and it's a very active research topic in parallel to text-to-image generation. There is also a potential for combining the two, but research into this is in early stages.

[carpathianpsychonaut]

Midjourney does allow a picture as base reference - you just add the URL to it as the first part of the prompt.

It also has much cheaper levels too like $10 for 200 gpu/min per month - fair to mention this for a like-for-like comparison

[ethan-fel]

the fact that SD is open-source and you can train it to your own use is kinda amazing. It's also pretty easy to install (quite trivial with docker). DALL-E and midjourney are nice but SD won the war already imo. Model like Waifu-diffusion already showcase the potential of community driven models.

[Phreno_Xeno]

The amount of low effort shovelware this technology will be used for makes me want to skip to another timeline.

[dacentafielda]

I've been using Stable Diffusion since it was released a couple of weeks ago. It's very fun to play with and see what you can get the ai to generate. Best for concept art and brainstorming.

[devvie_hu]

Since MidJourney is free (25 images max), it's the best one to play around with, just need to invite their bot to one of your Discrod servers, then make as many Discord accounts as you can (it's easy with the "dot in email address" trick).
It also supports image prompting actually, so it's quite versatile. 👍

[Michaelpstanich]

I've been working on a project and saw stable diffusion go up that 2 weeks ago and gave it a shot. While I don't think it will be useful for 'most' game art assets it's INSANELY GOOD for inspiration and idea generation. We as humans are very limited on what we can think of based on our mood, experiences, current environment, ect. This AI tech can give us experiences and visuals which we would never otherwise think of. Stable Diffusion has a parameter to allow the AI to 'consider' our prompt more or less, starting at more I'd generate 25-30 images, then lower it and generate another 25-30. The variation is huge and was insanely eye opening to a potential I had never even seen talked about before.

I don't believe AI will replace human artists as games tend to have very specific artistic goals, it may help in generic textures like brick walls and other tiling textures used in a lot of games, but it will no doubt be revolutionary in the brainstorming, concepting, and idea generation phases of game development.

[Skylancer]

It's really impressive stuff. Though I will say it is kinda sad to see how this will affect those in the art world which is already a grueling field that tends to go underpaid. Just shows though that when people said "AI will never beat humans at (insert job here)", they were wrong.

[ooiiooiiooii]

“…with minimal human input” except for the countless amount of stolen artwork they pumped into these A.I. to train them to even do this.

[NickGeo]

You should check out NERF as well. It makes 3D modeling from photos much faster

[esaedvik]

Now we need an artist AI that uses prompts for an AI to creatively make art...

[nullunit]

Midjourney, Stable Diffusion, and these other AI suites are stealing the effort and labor of artists whom spent decades building their skillset and growing into a style. For building textures or more general images it seems safe but even in the intro, when it was building that prompt for a snowy cabin, the output looked like bits of Thomas Kincaid paintings mashed together. That dude was a boring artist but it was identifiably similar to his style if you know who he is.

If it was solely artists or companies curating their own art into an AI and using prompts to streamline their workflow, that would be a better use but right now it is troubling that the best results I have seen are when an artists name is part of the prompt which guarantees that the dataset scraped someone elses work without their permission. I wish they didn't just indiscriminately scrape the internet for art to feed the AI so it could pump out knock off art. Even in these prompts I see "Artstation" as a modifier a lot and that is just stealing art from whatever art was scraped from into the Dataset from Artstation. Also, I see people romanticizing prompts as art when it is just stealing; there is no creative output without someone elses labor being stolen.

Cool tool though, it will continue to grow in use, I just hope they find a way to credit and compensate the people whose art is being stolen.

[Kotka]

Yeah, game dev opinion here - not really sure about these apps being "revolutionary for game devs". There are some good practical usage - could be faster creation of tileable textures out of photos ( although in that case, the creation of normal map, roughness & so on would have to rely on lower quality conversion from albedo info instead of way more precise maps from Adobe Substance 3D or Zbrush.)One downside is that you almost always still get that AI-upscaled " watercolor" look, if you zoom in those. The other obvious & already implemented usage is pretty good AI -upscaling of textures of older games or when porting a game to a more powerful platform. But If we talk about faster iteration of concept art - sure, you get sometimes awesome visual results, some interesting shapes, lighting, and a lot of variations, but you lose a lot of the key purpose of the production concept art - to provide the game devs pretty specific mood & idea and art direction. Not to mention unique compositions and object placement that are needed in order to better approach the realization of these environments & art direction from real -time graphics limitations / level & game design point of view. The current mess of people directly using as prompts the artwork of famous digital/traditional artists ( and pissing of a lot of those artists in the process ) and photos that can have their own licensing quirks ( gettys images ) also sound like too much of a legal / licensing nightmare to be used extensively .I think there is definitely potential and interesting usage cases, but not as major to the game development process.

[SquintyGears]

i love the technology but the unfortunate reality is going to be that loads of artist jobs are going to be removed from studios, massively reducing the development costs, but the consumers won't see games become any cheaper. the executives will just pocket the difference

[jamesrinley]

As a former "artist" (had to produce some web layouts) I'd like to give props for that last segment of the video. Great video, concerning tech.