Enhance! AI Super Resolution Is Here!

📝 The paper "Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wild" is available here:

📝 My latest paper on simulations that look almost like reality is available for free here:

Or this is the orig. Nature Physics link with clickable citations:

🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible:
Alex Balfanz, Alex Haro, B Shang, Benji Rabhan, Bret Brizzee, Gaston Ingaramo, Gordon Child, Jace O'Brien, John Le, Kyle Davis, Lukas Biewald, Martin, Michael Albrecht, Michael Tedder, Owen Skarpness, Richard Putra Iskandar, Richard Sundvall, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Tybie Fitzhugh, Ueli Gallizzi.

[juhor.]

The use of negative prompts if pretty clever. Sort of like using an adverserial network.

[Mushbee]

This looks more like gen fill than upscaling to be honest. Just check out the car example, the manufacturer logo chaanged completely even tough there was enough information to at least rebuild into some similar shape.

[bobclarke]

These should be called "Re-imaginings" rather than "Enhancement"

[foolwise]

This can become a real problem when it comes to truthfulness of data though. Imagine this being applied to surveillance cameras used to identify criminals. The ai is likely to extrapolate faces to common features rather than recognizable, true ones.

[gaweyn]

what was missing: taking a high res photo (ground truth), creating a low res version that is processed by this model (SUPIR), finally comparing SUPIR to ground truth. Otherwise very promising paper.

[jackmg]

Agreed, the ai is making it up the best it can from reference it has so you are essentially looking a new render rather than enchanted image

[fen]

I love how the triangle badge turned into a Nissian lol. AI: Close enough.

[Darhan]

You can't resolve information that's not actually present in the pixelated image. You *can* allow the model to make a reasonable guess and come up with something visually convincing. If there are several images in a series that the model can draw from for context, then presumably you can make the "upscaled image" more accurately represent what was originally there. But that probably requires another paper. The idea of using text prompts to help it understand what the missing information is is sort of a half-way measure to doing that.

[Razumen]

Can't say this is something we'd want to use regularly from images on the net, considering how it basically does hallucinate details that aren't there.

[brianhauk]

Holy mother of papers! I agree and look forward to seeing appropriate use of this technology preceded by societal discussion to define what's appropriate and what is not.

[percy]

I don't understand why you'd imply this AI up scaling could possibly 'guess' the number plate.
It has like 4 blurry lines. The number plate doesn't even properly overlap the original image.

[davidvincent]

Training models with bad samples associated with negative prompts is such a good idea, you wonder why it hasn't been done before. And that method can apply to all kind of generative AIs

[lobabobloblaw]

It is indeed a fantastic model. Can’t wait for motion implementations.

[jonbrooks]

New two minute paper! Thank you for keeping us informed Dr Károly Zsolnai-Fehér!

[scourgehh]

I can't wait u til this becomes open to thr public. Would be so helpful

[holahandstrom]

Looking forward to see it used on Space telescopes images and in phones.
... or those always blurry UFO images ;D ... I might be used as a pre-warp signature - "Can they see us ? Nop. We'll come back later."
Seriously, great progress! I guess video is the next step.

[nassifsamuel]

Amazing, so many applications for this upscaling

[mariokotlar]

You should have mentioned that this works by using SDXL (Stable Diffusion XL).
I cite the paper: Specifically, SUPIR employs StableDiffusion-XL (SDXL) [63] as a powerful generative prior, which contains 2.6 billion parameters. To effectively apply this model, we design and train a adaptor with more than 600 million parameters.

The only thing holding us back from using SDXL like this ourselves to upscale to very high resolutions without server grade GPUs is the lack of ControlNet tile model for it.

[fiddley]

Love to see some of the old Movies through this to see what they might look like if done today.

[vectoralphaSec]

Imagine ALL the low resolution 240p, 480p and lower movies, tv shows and videos from the past fully restored to basically 4K with this AI. That would be incredible. I would love to watch SD shows, and cartoons and anime from back in the day with this clarity or old 1940s and 50s black and white movies in 4K and colorized. The near future will be amazing.