Deep Bilateral Learning for Real-Time Image Enhancement

Performance is a critical challenge in mobile image processing. Given a reference imaging pipeline, or even human-adjusted pairs of images, we seek to reproduce the enhancements and enable real-time evaluation. For this, we introduce a new neural network architecture inspired by bilateral grid processing and local affine color transforms. Using pairs of input/output images, we train a convolutional neural network to predict the coefficients of a locally-affine model in bilateral space. Our architecture learns to make local, global, and content-dependent decisions to approximate the desired image transformation. At runtime, the neural network consumes a low-resolution version of the input image, produces a set of affine transformations in bilateral space, upsamples those transformations in an edge-preserving fashion using a new slicing node, and then applies those upsampled transformations to the full-resolution image. Our algorithm processes high-resolution images on a smartphone in milliseconds, provides a real-time viewfinder at 1080p resolution, and matches the quality of state-of-the-art approximation techniques on a large class of image operators. Unlike previous work, our model is trained off-line from data and therefore does not require access to the original operator at runtime. This allows our model to learn complex, scene-dependent transformations for which no reference implementation is available, such as the photographic edits of a human retoucher.

[lakloplak]

Dutch newspaper NU.nl linked your video and wrote an article about it. Good work! Just thinking of the possibilities make me really excited!

[JorgeGamaliel]

Awesome!! There are a lot of civil and military applications.

[EEBADUGANIVANJARIAKANKSH]

The code is not present at the given link mentioned in the video, can you please share the exact location where the repository is moved to? BTW I really liked the way you presented it, the animation, and other suff.

[aalabasy]

It's looks great .. any idea when will be available for android as APK .. thank you

[EEBADUGANIVANJARIAKANKSH]

how to make such kind of videos ? (I meant the animation)

[carlosjuniorgalarce]

Wow! Awesome, when it will be available?

[leobonston]

1) Did you upload it to google play? What is it called?
2) Can I find this program for a PC or just a set of code?
Thank.

[willoveryou]

Hey. Are you still working on this? Will it have a beta test?
Great work!

[arpanojha]

Hi Michael, is there any way i can get associated with the project ? I have trained classifiers for ANPR .

[JaspreetSingh-cbem]

This location points to which location ?? Please help ?

[kodecoder]

Cannot wait to see this in real work, it'd be great to see this hands-on, when would apk/code be available?

[AlexMetslov]

Who the hell didn't like this vid???

[ArseySOAD]

Is there a paper for it? How guidance maps work, where can I read about it?

[dailymail]

Hi Michael, can DailyMail.com post this video (or part of it) through our player, with a credit and link back to your YouTube channel? This would be to use in an article we're writing about this amazing project, as well as our social media posts about it. Would this be possible? Please let us know! You can reach us here or at nyvideo (at) assocnews (dot) co (dot) uk. Thanks so much!