Projector Compensation Framework Using Differentiable Rendering

Jino Park , Donghyuk Jung , Bochang Moon

Gwangju Institute of Science and Technology

IEEE Access

Projector Compensation Framework Using Differentiable Rendering
Projection mapping results of the state-of-the-art (CompenNet++) and our method (ours) given a non-planar and colored projection surface. The blue-colored box (in (a)) denotes the target display area where we want to show the target (ground truth) image (d) on the surface. Both methods estimate optimal projector input images ((b) and (c)) for this projection mapping task by adjusting its shape and colors. Our framework optimizes the input image using differentiable rendering and produces a more accurate output image (f) than the state-of-the-art result (e).

Abstract

Projection mapping is a widely adopted technique in various applications. A typically projector-camera system consists of a projector and camera pair where a projector emits an application-specific input image onto a surface, and the camera captures the projected output image. Given an ideal projection surface with white-colored planar geometries, the camera (or a viewer) can observe the projected output image without any visual distortion. The surface, however, can have arbitrary shapes and textured colors in practice, and it often introduces visually distracting artifacts to the output image. It results in lowering the viewing experience drastically. We propose a projector compensation framework that adjusts the projector input image so that the camera can see the projected output image with a much-reduced distortion. Our key contribution is to model the real projection mapping process with a virtual but controllable light simulation and optimize the projector input using differentiable rendering. We demonstrate that our new framework produces a more accurate output than state-of-the-art methods given complex projection surfaces.

Contents

BibTex