Available at: https://digitalcommons.calpoly.edu/theses/2795
Date of Award
6-2024
Degree Name
MS in Computer Science
Department/Program
Computer Science
College
College of Engineering
Advisor
Jonathan Ventura
Advisor Department
Computer Science
Advisor College
College of Engineering
Abstract
We propose Instant High Dynamic Range Neural Radiance Fields (Instant HDR-NeRF), a method of learning high dynamic range (HDR) view synthesis from a set of low dynamic range (LDR) views with unknown and varying exposure and white balance in as little as minutes. Our method can render novel HDR views without ground-truth supervision, and novel LDR views in different exposure settings, including those that match the ground-truth LDR views. The key to our method is to model the physical process of the camera with two implicit MLPs: a radiance field and a monotonically increasing tone-mapper. Built upon Instant Neural Graphics Primitives (Instant-NGP), the radiance field encodes the scene geometry and radiance (from 0 to ∞), and outputs the densities and the radiance at locations along the camera ray. The monotonically increasing tone-mapper models the camera response function (CRF) where the radiance hits on the camera sensor and becomes a pixel value (from 0 to 255). The radiance at each location is combined with the learnable exposure parameters, optimized separately for each color band and for each image. A quantitative evaluation on benchmark datasets shows that our method outperforms prior HDR novel view synthesis methods in LDR rendering quality and training speed. To best of our knowledge, our method is also the first HDR radiance field that successfully recovers the ground-truth CRF with a low average error rate of 3.70%, while co-learning geometry, radiance, and exposures all at the same time through implicit functions. In practical applications, our method can produce high-fidelity 3D reconstruction of real-world scenes from images of varying exposure settings, which is particularly useful for casual capturing, where fixed settings aren’t guaranteed. The tone-mapper MLP can be easily controlled to simulate auto-exposure effects, making it useful in filming and video games. Furthermore, the HDR radiance maps produced by our method can be edited and tone-mapped according to user preferences.