6DOPE-GS: Online 6D Object Pose Estimation using Gaussian Splatting

Efficient and accurate object pose estimation is crucial for applications such as AR, autonomous driving, and robotics. While model-based 6D pose methods show strong performance, model-free approaches often suffer from high computational cost in real-time RGB-D settings. We introduce 6DOPE-GS, a method for online 6D object pose estimation and tracking using a single RGB-D camera. It leverages fast differentiable rendering via Gaussian Splatting to jointly optimize object pose and 3D reconstruction in real time. To improve robustness and speed, our approach combines incremental 2D Gaussian Splatting with dynamic keyframe selection and opacity-based pruning. This ensures high spatial coverage and adaptive Gaussian control. Experiments on HO3D and YCBInEOAT demonstrate that 6DOPE-GS achieves performance comparable to state-of-the-art model-free baselines while offering a 5× speedup, enabling efficient tracking and reconstruction in real-world scenarios.

6DOPE-GS begins by segmenting the target object using SAM2 in the first RGB-D frame, and tracks it across the sequence. Keyframes are selected via LoFTR correspondences and initialized with coarse poses using bundle adjustment. These keyframes are jointly optimized with 2D Gaussians through differentiable rendering for accurate pose refinement and object reconstruction. A dynamic keyframe selection strategy improves spatial coverage, while an adaptive Gaussian pruning mechanism ensures efficiency. The final keyframe poses guide an online pose graph for continuous tracking during runtime.