4D volumetric video via Gaussian Splatting (GS) enables free viewpoint control of real-world scenes but does not enable motion editing. One challenge is achieving part-based control in multi-object scenes; prior works focus on single objects, assume rigid parts, or require manual annotations.
MotionSplicer is a template-free and annotation-free method that automatically discovers 3D parts in GS. To bridge the semantic-to-motion domain gap, it extracts initial parts from temporal difference images and resolves multi-view inconsistencies via an adaptive 3D unify-and-split process. For robust tracking, it optimizes a time-invariant feature grid with temporal refinement to learn spatially and temporally coherent skinning weights.
Experiments show that MotionSplicer can be applied more generally to objects and scenes than state-of-the-art methods, and enables creative motion edits like isolation, ghosting, and echo.
MotionSplicer models the scene with 3D Gaussians carrying part weights that move via linear blend skinning. A motion-aware 4D part discovery framework extracts motion handles from raw video without annotations, by lifting 2D segmentations of temporal difference images and applying a novel 3D unification-and-splitting process. Then, a continuous spatio-temporal optimization learns skinning weights via a time-invariant feature grid coupled with temporal consistency refinement, enforcing spatio-temporally coherent motion tracking for complex, non-rigid parts.
MotionSplicer automatically discovers parts without templates or manual annotations. Below we visualize the optimized skinning weights (top row) and the optimized handles (bottom row) across diverse objects.
Once parts are discovered, previously impractical edits can be performed on a 4D volumetric video simply by selecting a part handle and adjusting its timing or motion: loop, isolation, echo, ghosting, speed, freeze, compositing, and (optionally spatial) scaling.
We build an interactive viewer for 4D motion editing, supporting per-part visualization, motion isolation, freeze, ghosting, echo, and optional 4D spatial editing. All code, including the viewer, will be open-sourced. Stay tuned!
4D motion editing
4D spatial editing
MotionSplicer handles complex multi-object scenes and non-rigid parts where rigid-articulation, rig-based, optimization-only, and table-top approaches struggle.
Kitchen doors: motion isolation
This work was supported by NASA grant #80NSSC23M0075 and ONR DURIP grant N00014-23-1-2804.