Teleoperation could be used to replace a backup safety driver on autonomous vehicles and could play a valuable role in scaling up deployment of autonomous vehicles. The surrounding environment of the remote vehicle needs to be recreated for the teleoperator using video images received over wireless channels. To handle the significant time delays in receiving remote video data, this paper develops a predictive display system that uses deep learning to estimate the current video display for the teleoperator from old (delayed) camera images. First, old camera and Lidar data are fused to create a 3D reconstruction of the remote vehicle environment using conventional and deep-learning-based algorithms. Then the ego-vehicle’s real-time position and orientation variables are estimated using an extended Kalman filter. Predictive modification of the reconstructed old 3D scene is performed using the ego-vehicle’s estimated new trajectory variables. Deep-learning based image in-painting is used to improve image quality. Furthermore, this paper also introduces a new image comparison metric for evaluating the accuracy of the object detection and localization performance in the predictive display image. Real-world experimental data from the nuScenes and Kitti datasets are used for evaluation of the proposed system. The predictive display images are compared with ground truth images using various image comparison metrics and shown to provide significantly superior performance compared to the actual delayed images received over wireless channels.

Phase retrieval (PR) is fundamentally important in scientific imaging and is crucial for nanoscale techniques like coherent diffractive imaging (CDI). Low radiation dose imaging is essential for applications involving radiation-sensitive samples. However, most PR methods struggle in low-dose scenarios due to high shot noise. Recent advancements in optical data acquisition setups, such as in-situ CDI, have shown promise for low-dose imaging, but they rely on a time series of measurements, making them unsuitable for single-image applications. Similarly, data-driven phase retrieval techniques are not easily adaptable to data-scarce situations. Zero-shot deep learning methods based on pre-trained and implicit generative priors have been effective in various imaging tasks but have shown limited success in PR. In this work, we propose low-dose deep image prior (LoDIP), which combines in-situ CDI with the power of implicit generative priors to address single-image low-dose phase retrieval. Quantitative evaluations demonstrate LoDIP's superior performance in this task and its applicability to real experimental scenarios.