Deep Bilateral Learning for Real Time Image Enhancement

Performance is a critical challenge in mobile image processing. Given a reference imaging pipeline, or even humanadjusted pairs of images, we seek to reproduce the enhancements and enable realtime evaluation. For this, we introduce a new neural network architecture inspired by bilateral grid processing and local affine color transforms. Using pairs of input, output images, we train a convolutional neural network to predict the coefficients of a locallyaffine model in bilateral space. Our architecture learns to make local, global, and contentdependent decisions to approximate the desired image transformation. At runtime, the neural network consumes a lowresolution version of the input image, produces a set of affine transformations in bilateral space, upsamples those transformations in an edgepreserving fashion using a new slicing node, and then applies those upsampled transformations to the fullresolution image. Our algorithm processes highresolution images on a smartphone in milliseconds, provides a r