Research

Problem. Feed-forward 3D foundation models made reconstruction fast and robust, but they answer with a single estimate and no measure of confidence. Downstream systems such as sensor fusion, active vision, and robotics need to know how far to trust each pose, not only what the pose is.

Idea. Keep VGGT’s backbone and pose prediction frozen, and train a second lightweight branch to predict a full $6 \times 6$ covariance for each camera pose, formulated on $\mathrm{SE}(3)$. The branch uses body-centric perturbations on the Lie algebra, a left-invariant metric that reconciles meters with radians, and a negative log-likelihood with a curriculum that prevents the covariances from collapsing.

Why it matters. On CO3D the predicted uncertainty ranks frames by their pose error and its structure matches the analytical covariance of classical bundle adjustment; the fitted temperature is 1.01, so the head is already calibrated on typical frames. On EPIC-KITCHENS it reports that a dynamic reconstruction is unreliable as a whole, though it does not single out the individual frames that are misplaced. This turns an otherwise deterministic reconstructor into a component that can be used in probabilistic pipelines, without modifying its frozen pose predictor.

Problem. A prosthetic hand controlled by EEG receives a noisy, low bandwidth signal of the user’s intent. Before the hand closes around an object, something has to check whether the grasp is actually feasible, in real time and from a single camera.

Idea. Combine object detection, monocular depth, segmentation, and hand-pose tracking into a Kalman-filtered 3D state for both the hand and the target object. A validation model then judges the intended grasp using the object’s 3D geometry and the hand pose together.

Why it matters. Grasp validation is the safety layer between the user thinking about a grasp and the motors moving. The pipeline runs live, ships with a graphical demo and a Docker setup, and the report is public.