Enhancing Photorealism Enhancement

Enhancing Photorealism Enhancement
Stephan R. Richter, Hassan Abu AlHaija, and Vladlen Koltun

We present an approach to enhancing the realism of synthetic images. The images are enhanced by a convolutional network that leverages intermediate representations produced by conventional rendering pipelines. The network is trained via a novel adversarial objective, which provides strong supervision at multiple perceptual levels. We analyze scene layout distributions in commonly used datasets and find that they differ in important ways. We hypothesize that this is one of the causes of strong artifacts that can be observed in the results of many prior methods. To address this we propose a new strategy for sampling image patches during training. We also introduce multiple architectural improvements in the deep network modules used for photorealism enhancement. We confirm the benefits of our contributions in controlled experiments and report substantial gains in stability and realism in comparison to recent image-to-image translation methods and a variety of other baselines.

[IOwnASUV]

How is it photoreal if there isn’t 600 gallons of water on the ground

[AktienMitKopf]

I absolutely understood nothing, but it looks great

[da_pikmin_coder]

It's crazy how the photo-realistic version actually has *less* details, but looks more real.

[kohakkanuva]

I think the biggest roadblock to good photorealism is that sometimes people don’t realize that reality isn’t always so beautiful or crisp. A 4K texture is just a texture. Depending on distance in reality, colors meld together, and everything isn’t so bright. It’s the imperfections that make something look photoreal, not the lack of

[FBCxUNKNOWN]

For once a realism mod that doesn’t just add million puddles and reflections

[robinrai]

"It looks dull and like a dashcam"

Yeah no shit that's what it was trained on

[fattabby]

If you show this and tell me it is a dash cam footage, I wouldn't even recognize

[rsjonm]

This is the most realistic gameplay we'll ever see. Imagine future engines doing this in real time.

[madmax-fnqq]

finally a realistic graphics mod without making the floor wet

[Frozen.wasser]

So this is what GTA IV's color grading was trying to achieve..

[Mac-nwuc]

it looks like a dashcam that has a windscreen obscuring it

[robbylebotha]

If a AAA studio released a game with your realism filter, game streamers would call it dull.

[sebYB]

The most impressive part about this is that they're playing GTA V while actually respecting traffic laws

[FPSNecromancerBob]

So this is why the matrix had a green tint

[ChengTeoh]

I would love to see how this program "enhances" the NPC characters walking around in the world.
What do their faces look like when looking at it close up after being trained.

[TurboLight]

The comparation with the Los Angeles dataset really puts in perspective of how well Rockstar captured the Los Angeles atmosphere in GTA V

[FriedFrenchFries]

I was expecting a bad mod showcase but I'm actually impressed by this.

[arandomsupra]

The only thing holding this back is that you feel slightly more guilty when you kill a random pedestrian

[VinaXR]

Game developers: "let's make this game look detailed and colorful to be realistic"
Some German Guy: "realism is Russian dashcam video"

[ToriKo_]

Wow. I love the version trained on typical dashcams. I have always said I’d love to play a game that had realistic graphics, not high res, or ray traced etc. But actually realistic graphics, even if it looks low quality like a dash cam. Amazing work