We reframe scene flow as the problem of estimating a continuous space and time PDE that describes motion for an entire observation sequence, represented with a neural prior. Our resulting unsupervised method, EulerFlow, produces high quality scene flow on real-world data across multiple domains, including large-scale autonomous driving scenes and dynamic tabletop settings. Notably, EulerFlow produces high quality flow on small, fast moving objects like birds and tennis balls, and exhibits emergent 3D point tracking behavior by solving its estimated PDE over long time horizons. On the Argoverse 2 2024 Scene Flow Challenge, EulerFlow outperforms all prior art, beating the next best unsupervised method by over 2.5x and the next best supervised method by over 10%.
We present representative examples of EulerFlow’s performance on a variety of scenes in order to provide honest depictions of the performance you can expect out of EulerFlow on diverse, in-the-wild data. While these results highlight EulerFlow’s many strengths, they also accurately portray its weaknesses, such as adding motion to partially occluded areas or on nearby static surfaces. Scenes with RGB information are for visualization purposes only; RGB information is not used in the flow estimation process.
@article{vedder2024eulerflow,
author = {Vedder, Kyle and Peri, Neehar and Khatri, Ishan and Li, Siyi and Eaton, Eric and Kocamaz, Mehmet and Wang, Yue and Yu, Zhiding and Ramanan, Deva and Pehserl, Joachim},
title = {{Scene Flow as a Partial Differential Equation}},
year = {2024},
journal = {arXiv preprint arXiv:2410.02031},
website = {https://vedder.io/eulerflow},
pdf = {https://arxiv.org/abs/2410.02031}
}