RefracGS: Novel View Synthesis Through Refractive Water Surfaces with 3D Gaussian Ray Tracing

Novel view synthesis (NVS) through non-planar refractivesurfaces presents fundamental challenges due to severe, spatially varyingoptical distortions. While recent representations like NeRF and 3D Gaus-sian Splatting (3DGS) excel at NVS, their assumption of straight-lineray propagation fails under these conditions, leading to significant arti-facts. To overcome this limitation, we introduce RefracGS, a frameworkthat jointly reconstructs the refractive water surface and the scene be-neath the interface. Our key insight is to explicitly decouple the refractiveboundary from the target objects: the refractive surface is modeled via aneural height field, capturing wave geometry, while the underlying sceneis represented as a 3D Gaussian field. We formulate a refraction-awareGaussian ray tracing approach that accurately computes non-linear raytrajectories using Snell’s law and efficiently renders the underlying Gaus-sian field while backpropagating the loss gradients to the parameterizedrefractive surface. Through end-to-end joint optimization of both rep-resentations, our method ensures high-fidelity NVS and view-consistentsurface recovery. Experiments on both synthetic and real-world sceneswith complex waves demonstrate that RefracGS outperforms prior re-fractive methods in visual quality, while achieving ∼15× faster trainingand real-time rendering at 200 FPS.