Online Neural Denoising with Cross-Regression for Interactive Rendering, Computer Graphics Lab. @ GIST

Online Neural Denoising with Cross-Regression for Interactive Rendering

Hajin Choi ¹ , Seokpyo Hong ² , Inwoo Ha ^{2, 3} , Nahyup Kang ² , Bochang Moon ¹

Gwangju Institute of Science and Technology ¹ , Samsung Advanced Institute of Technology ² , KAIST ³

ACM Transactions on Graphics (SIGGRAPH Asia 2024)

Online Neural Denoising with Cross-Regression for Interactive Rendering

Denoising results of a regression-based denoiser (BMFR [Koskela et al. 2019]) and our method for an interactive path tracing framework (ReSTIR PT [Lin et al. 2022]). While the existing regression using G-buffers, e.g., textures and normals, preserves geometric edges, it tends to blur other image details (e.g., shadows) that the G-buffers cannot capture. We present a hybrid denoising framework that employs local regression with a neural network for robust denoising capable of maintaining such non-geometric edges. We modify the regression into a cross-regression form to generate pilot estimates, which serve as input to our neural network and guide its online training using only runtime image sequences. 3D model courtesy of [Lumberyard 2017].

Abstract

Generating a rendered image sequence through Monte Carlo ray tracing is an appealing option when one aims to accurately simulate various lighting effects. Unfortunately, interactive rendering scenarios limit the allowable sample size for such sampling-based light transport algorithms, resulting in an unbiased but noisy image sequence. Image denoising has been widely adopted as a post-sampling process to convert such noisy image sequences into biased but temporally stable ones. The state-of-the-art strategy for interactive image denoising involves devising a deep neural network and training this network via supervised learning, i.e., optimizing the network parameters using training datasets that include an extensive set of image pairs (noisy and ground truth images). This paper adopts the prevalent approach for interactive image denoising, which relies on a neural network. However, instead of supervised learning, we propose a different learning strategy that trains our network parameters on the fly, i.e., updating them online using runtime image sequences. To achieve our denoising objective with online learning, we tailor local regression to a cross-regression form that can guide robust training of our denoising neural network. We demonstrate that our denoising framework effectively reduces noise in input image sequences while robustly preserving both geometric and non-geometric edges, without requiring the manual effort involved in preparing an external dataset.