Spatio-temporal forecasting is crucial in various fields and requires a careful balance between identifying subtle patterns and filtering out noise. Vector quantization (VQ) appears well-suited for this purpose, as it quantizes input vectors into a set of codebook vectors or patterns. Although VQ has shown promise in various computer vision tasks, it surprisingly falls short in enhancing the accuracy of spatio-temporal forecasting. We attribute this to two main issues: inaccurate optimization due to non-differentiability and limited representation power caused by hard-VQ structure. To tackle these challenges, we introduce Differentiable Sparse Soft-Vector Quantization (SVQ), the first VQ method to enhance spatio-temporal forecasting. SVQ balances detail preservation with noise reduction, offering full differentiability and a solid foundation in sparse regression. The method employs a two-layer MLP and an extensive codebook to streamline the sparse regression process, significantly cutting computational costs while simplifying training and improving performance. Empirical studies on five spatio-temporal benchmark datasets show SVQ achieves state-of-the-art results, including a 7.9% improvement on the WeatherBench-S temperature dataset and an average mean absolute error reduction of 9.4% in video prediction benchmarks (Human3.6M, KTH, and KittiCaltech), along with a 17.3% enhancement in image quality (measured by LPIPS).

Adverse weather conditions like rain, fog, and snow significantly hinder perception in autonomous driving systems. This paper proposes a multimodal contrastive learning and transfer learning-based adaptive image restoration method for multiple adverse weather conditions. By integrating image and textual information, our method enhances robustness to diverse weather scenarios. Specifically, we first fine-tune a contrastive language-image pre-trained model to develop a multimodal image classifier capable of recognizing adverse weather conditions. Subsequently, an encoder-decoder-based restoration network is employed, where cross-attention layers incorporate textual conditional information, enabling the network to perceive weather variations. An adaptive restoration strategy is then applied to target specific noise characteristics associated with different weather conditions. Experiments on Rain Cityscapes, Foggy Cityscapes, and Snow Cityscapes show our model outperforms task-specific and All-in-One methods in visual and real-time performance, providing an efficient and robust solution for autonomous driving in complex environments.

This paper presents a collision avoidance system based on human-robot interaction for mobile robots, which are working alongside to humans. In the future work environments, mobile robots will work side by side to humans. This raises big challenges related to the safety of the human and the ability of the robot to identify the people, interact with them and avoid physical accidents to them. To realize these concepts, a human-robot interaction system together with a collision avoidance system are developed to enable a safe navigation of the mobile robots. In this system, when a robot meets people in its path, it will try to interact with the human to execute one of the three actions: move forward, move backward, and collision avoidance; or the robot will implement the collision-avoidance autonomously when no people interacted with it. The interaction is based on the gestures obtained from the Kinect 2.0 sensor, and the system was tested using a H20 Robot (Canada). The experimental results proved the validity of the proposed system in interacting with the humans, and avoiding them.

In this paper, a new human feature based method is proposed for the intelligent HMRI (Human-Mobile Robot Interaction) in the indoor life science laboratories. The proposed method includes the contents as: (a) the Microsoft Kinect Sensors equipped on the mobile robots are adopted to detect the human and measure their face color images; (b) the different face features in the measured face images are defined to recognize the dynamic human face orientations. To find the best one among the available features, their comparison is provided, including the eyebrow zone, the eye zone, the hybrid eyebrow-eye zone and the nose zone; (c) the Extreme Learning Machine (ELM) is built to complete the intelligent computation for the robust recognition of the face orientations; and (d) based on the recognizing results of the face orientations, the human face moving directions are obtained successfully. Since the proposed HMRI strategy is developed for the real-time mobile robot based transportation, the computational accuracy and the real-time performance are both focused in this study. The experimental results indicate that the nose feature based HMRI strategy has the best performance, which can reach the success rate of 99.89% only consuming 0.127s.