Target-Aware Image Denoising for Inverse Monte Carlo Rendering

Jeongmin Gu 1 , Jonghee Back 1 , Sung-Eui Yoon 2 , Bochang Moon 1

Gwangju Institute of Science and Technology 1 , KAIST 2

ACM Transactions on Graphics (SIGGRAPH 2024)

Target-Aware Image Denoising for Inverse Monte Carlo Rendering
Optimization results where we use a gradient-based optimizer that infers the scene parameters (i.e., textures within the yellow box) from its initial (a reddish one) so that the rendered image with the inferred textures is close to the target image. We compare the images rendered using the parameters inferred by the inverse rendering optimization without and with image denoising, i.e., the baseline and two denoisers (a cross-bilateral filter and our denoiser). Adopting an existing denoiser (i.e., the cross-bilateral filter) allows faster convergence than the baseline without image denoising, but the optimization goes into an undesirable local minimum, i.e., over-blurred textures. On the other hand, our image denoiser makes the scene inference robust by guiding the optimizer to preserve the texture details.

Abstract

Physically based differentiable rendering allows an accurate light transport simulation to be differentiated with respect to the rendering input, i.e., scene parameters, and it enables inferring scene parameters from target images, e.g., photos or synthetic images, via an iterative optimization. However, this inverse Monte Carlo rendering inherits the fundamental problem of the Monte Carlo integration, i.e., noise, resulting in a slow optimization convergence. An appealing approach to addressing such noise is exploiting an image denoiser to improve optimization convergence. Unfortunately, the direct adoption of existing image denoisers designed for ordinary rendering scenarios can drive the optimization into undesirable local minima due to denoising bias. It motivates us to reformulate a new image denoiser specialized for inverse rendering. Unlike existing image denoisers, we conduct our denoising by considering the target images, i.e., specific information in inverse rendering. For our target-aware denoising, we determine our denoising weights via a linear regression technique using the target. We demonstrate that our denoiser enables inverse rendering optimization to infer scene parameters robustly through a diverse set of tests.

Contents

License

This work is licensed under a Creative Commons Atribution International 4.0 License.

Creative Commons Atribution International 4.0 License