Reconstructing outdoor 3D scenes from temporal observations is a challenge that recent work on neural fields has offered a new avenue for. However, existing methods that recover scene properties, such as geometry, appearance, or radiance, solely from RGB captures often fail when handling poorly-lit or texture-deficient regions. Similarly, recovering scenes with scanning LiDAR sensors is also difficult due to their low angular sampling rate which makes recovering expansive real-world scenes difficult. Tackling these gaps, we introduce Gated Fields - a neural scene reconstruction method that utilizes active gated video sequences. To this end, we propose a neural rendering approach that seamlessly incorporates time-gated capture and illumination. Our method exploits the intrinsic depth cues in the gated videos, achieving precise and dense geometry reconstruction irrespective of ambient illumination conditions. We validate the method across day and night scenarios and find that Gated Fields compares favorably to RGB and LiDAR reconstruction methods. Our code and datasets are available here11https://light.princeton.edu/gatedfields/.

Multi-modal sensor fusion constitutes an essential ingredient for safe autonomous navigation. In the last years, many works have improved the accuracy of Deep-Learning-based odometry estimators. However, the robustness of these algorithms to sensor failure or measurement degradation, which are very likely to happen during navigation, has been studied less extensively. Furthermore, works studying the robustness of the fusion modules are developed without modeling the correlation between sensor features, which is crucial to filter out features derived from noisy measurements and in sensor faults scenarios. To bridge this gap, in this paper, we propose a fault-resistant odometry estimator, which produces robust estimates even when the sensors completely fail, or measurements progressively degrade. Our framework models the correlation between the sensor embedding using Message Passing Neural Network (MPNN), a particular type of Graph Neural Network (GNN). A mask is then computed from the updated node features of the graph to weigh the multi-modal features computed from different sensors. We evaluate the proposed fusion strategy on the modified raw KITTI dataset with sensor degradation scenarios. Finally, we compare against state-of-the-art baselines based on trivial features concatenation and soft-fusion to demonstrate our method’s superiority in terms of accuracy and robustness to sensor degradation and failures.