CSS-BA: Gate Guided Column Space Search for Bundle Adjustment

Bundle adjustment (BA) remains a critical refinement mod-ule for image-based 3D reconstruction and continues to improve geomet-ric accuracy even in learning-based pipelines. However, in low-parallaxand near-rotational regimes, classical Schur-based Levenberg–Marquardt(LM) often becomes ill-conditioned and yields unreliable pose and cali-bration estimates. We propose Gate-Guided CSS-BA, a solver-side mod-ification of Schur-LM that preserves the classical BA objective and trust-region framework while constraining each update to a geometrically in-formed low-dimensional subspace. By integrating Column Space Search(CSS) with geometry-aware gating, the method stabilizes the Schur-LMupdate without altering the estimation problem. In contrast to keyframeor state-selection approaches, all camera and point parameters remainin the optimization problem; only the update direction is restricted. Themethod serves as a drop-in replacement for existing BA pipelines. Exper-iments on both generic and challenging weak-geometry scenarios showmore stable optimization, improved relative pose accuracy, and compet-itive calibration behavior while maintaining reprojection quality.