We present a method for dynamic surface reconstruction of large-scale urban scenes from LiDAR. Depth-based re-constructions tend to focus on small-scale objects or large-scale SLAM reconstructions that treat moving objects as outliers. We take a holistic perspective and optimize a compositional model of a dynamic scene that decomposes the world into rigidly-moving objects and the background. To achieve this, we take inspiration from recent novel view syn-thesis methods and frame the reconstruction problem as a global optimization over neural surfaces, ego poses, and object poses, which minimizes the error between composed spacetime surfaces and input LiDAR scans. In contrast to view synthesis methods, which typically minimize 2D errors with gradient descent, we minimize a 3D point-to-surface error by coordinate descent, which we decompose into reg-istration and surface reconstruction steps. Each step can be handled well by off-the-shelf methods without any re-training. We analyze the surface reconstruction step for rolling-shutter LiDARs, and show that deskewing operations common in continuous time SLAM can be applied to dynamic objects as well, improving results over prior art by an order of magnitude. Beyond pursuing dynamic reconstruction as a goal in and of itself, we propose that such a system can be used to auto-label partially annotated se-quences and produce ground truth annotation for hard-to-label problems such as depth completion and scene flow. Please refer https://anishmadan23.github.io/smore/ for more visual results.

Scene flow estimation is the task of describing the 3D motion field between temporally successive point clouds. State-of-the-art methods use strong priors and test-time optimization techniques, but require on the order of tens of seconds to process full-size point clouds, making them unusable as computer vision primitives for real-time applications such as open world object detection. Feedforward methods are considerably faster, running on the order of tens to hundreds of milliseconds for full-size point clouds, but require expensive human supervision. To address both limitations, we propose _Scene Flow via Distillation_, a simple, scalable distillation framework that uses a label-free optimization method to produce pseudo-labels to supervise a feedforward model. Our instantiation of this framework, _ZeroFlow_, achieves **state-of-the-art** performance on the _Argoverse 2 Self-Supervised Scene Flow Challenge_ while using zero human labels by simply training on large-scale, diverse unlabeled data. At test-time, ZeroFlow is over 1000$\times$ faster than label-free state-of-the-art optimization-based methods on full-size point clouds (34 FPS vs 0.028 FPS) and over 1000$\times$ cheaper to train on unlabeled data compared to the cost of human annotation (\\$394 vs ~\\$750,000). To facilitate further research, we will release our code, trained model weights, and high quality pseudo-labels for the Argoverse 2 and Waymo Open datasets.

Popular benchmarks for self-supervised LiDAR scene flow (stereoKITTI, and FlyingThings3D) have unrealistic rates of dynamic motion, unrealistic correspondences, and unrealistic sampling patterns. As a result, progress on these benchmarks is misleading and may cause researchers to focus on the wrong problems. We evaluate a suite of top methods on a suite of real-world datasets (Argoverse 2.0, Waymo, and NuScenes) and report several conclusions. First, we find that performance on stereoKITTI is negatively correlated with performance on real-world data. Second, we find that one of this task’s key components –removing the dominant ego-motion –is better solved by classic ICP than any tested method. Finally, we show that despite the emphasis placed on learning, most performance gains are caused by pre- and post-processing steps: piecewise-rigid refinement and ground removal. We demonstrate this through a baseline method that combines these processing steps with a learning-free test-time flow optimization. This baseline outperforms every evaluated method.

In this work, we explore a unique approach to harvest energy from active matter in order to independently power and control many untethered, mobile microrobots. More specifically, we investigate microstructure transport caused by stochastic collisions between freely swimming cells and rigid microstructure boundaries in a dense bacterial bath. We characterize the motion of asymmetrically shaped microstructures, specifically gears and chevrons, and demonstrate control of these microrobots by altering the motility of swimming cells with high intensity blue light. We demonstrate both rotational and translational motion control using a custom system for tracking and real-time automated exposure of local bath regions using visual feedback. These experiments are supported by a mathematical model and simulations which describe the dynamics of a microstructure propelled by many stochastic cell structure collisions. This model allows us to predict microrobot trajectories and to develop improved strategies for microrobot design and control.

Independent control of several, untethered microrobots remains a significant challenge. Although progress has been made in controlling multiple robots at scales below 100 mm, motion is dependent on external energy sources for propulsion and is often not fully decoupled. Here, we develop a method to dynamically pattern light at a resolution of 1 μm2per pixel in a closed loop real-time system using a digital micromirror device (DMD) system. We test the system on microbiorobots - bacteria-coated microplates - and demonstrate that we can selectively stimulate pre-defined regions of bioactuators. We verify previously developed dynamic models as a step toward developing individually addressable, self-propelled microrobots.