We reframe scene flow as the task of estimating a continuous space-time ODE that describes motion for an entire observation sequence, represented with a neural prior. Our method, EulerFlow, optimizes this neural prior estimate against several multi-observation reconstruction objectives, enabling high quality scene flow estimation via pure self-supervision on real-world data. EulerFlow works out-of-the-box without tuning across multiple domains, including large-scale autonomous driving scenes and dynamic tabletop settings. Remarkably, EulerFlow produces high quality flow estimates on small, fast moving objects like birds and tennis balls, and exhibits emergent 3D point tracking behavior by solving its estimated ODE over long-time horizons. On the Argoverse 2 2024 Scene Flow Challenge, EulerFlow outperforms all prior art, surpassing the next-best unsupervised method by more than 2.5x, and even exceeding the next-best supervised method by over 10%.
We present representative examples of EulerFlowā€™s performance on a variety of scenes in order to provide honest depictions of the performance you can expect out of EulerFlow on diverse, in-the-wild data. While these results highlight EulerFlowā€™s many strengths, they also accurately portray its weaknesses, such as adding motion to partially occluded areas or on nearby static surfaces. Scenes with RGB information are for visualization purposes only; RGB information is not used in the flow estimation process.
@article{vedder2024eulerflow,
author = {Vedder, Kyle and Peri, Neehar and Khatri, Ishan and Li, Siyi and Eaton, Eric and Kocamaz, Mehmet and Wang, Yue and Yu, Zhiding and Ramanan, Deva and Pehserl, Joachim},
title = {{Neural Eulerian Scene Flow Fields}},
year = {2024},
journal = {arXiv preprint arXiv:2410.02031},
website = {https://vedder.io/eulerflow},
pdf = {https://arxiv.org/abs/2410.02031}
}