Accurately estimating the 6-DoF pose of objects is a fundamental challenge in computer vision and robotics. While category-level pose estimation based on RGBD data has achieved good performance in recent years, estimating poses solely from RGB images remains a significant challenge. Existing RGB-based category-level methods primarily focus on recovering object point clouds from RGB images, and pose prediction is not performed end-to-end by a network. This paper presents a Category-level and Instance-level Pose Estimation Network (CIPE), which models pose estimation as a set prediction problem and enables direct pose regression from RGB images. To further enhance the network's ability to learn object poses, first, a novel learnable rotation representation that redefines rotation learning within Euclidean space is introduced to facilitate rotation regression. Additionally, we propose a prior-query fusion strategy that utilizes a pre-trained point cloud feature extraction network to integrate categorical object features with bounding boxes, thereby improving the incorporation of category information. Experimental results demonstrate that CIPE significantly outperforms existing RGB-based methods on both category-level and instance-level datasets. The code is available at https://github.com/jialeren/CIPE.

The open-vocabulary paradigm enhances 6D object pose estimation by leveraging language cues to transfer learned representations from seen to unseen objects, yet its performance suffers from a misalignment between vision-language representations and the 6D pose space. This challenge is compounded by the intrinsic lack of pose awareness in models like CLIP (Contrastive Language-Image Pre-Training). To address this limitation, we introduce CLIP-6D, which equips CLIP with the ability to estimate poses through the learning of generalizable representations specific to objects. CLIP-6D consists of three key components: (1) an innovative self-alignment strategy that enables CLIP to derive geometric representations from RGB images by leveraging its inherent feature extractor; % utilizing its inherent feature extraction capability; (2) a multiplex representation interactive learning method efficiently bridges the heterogeneous representations of object category priors, geometry, and spatial correlations; (3) a lightweight adapter using knowledge distillation improves CLIP's capture of detailed semantic representations. Experiments show that CLIP-6D achieves an improvement of 10.8% and 16.4% in the metric 5° 2 cm and 10° 5 cm for zero-shot generalization and achieves a speedup of 5.4 FPS in inference over current state-of-the-art methods. The source code and models are available at https://github.com/whoawong/CLIP-6D.git.

Semantic segmentation methods enhance robust and reliable understanding under adverse illumination conditions by integrating complementary information from visible and thermal infrared (RGB-T) images. Existing methods primarily focus on designing various feature fusion modules between different modalities, overlooking that feature learning is the critical aspect of scene understanding. In this paper, we propose a novel module-free Multiplex Interactive Learning Network (MiLNet) for RGB-T semantic segmentation, which adeptly integrates multi-model, multi-modal, and multi-level feature learning, fully exploiting the potential of multiplex feature interaction. Specifically, robust knowledge is transferred from the vision foundation model to our task-specific model to enhance its segmentation performance. In the task-specific model, an asymmetric simulated learning strategy is introduced to facilitate mutual learning of geometric and semantic information between high- and low-level features across modalities. Additionally, an inverse hierarchical fusion strategy based on feature learning pairs is adopted and further refined using multilabel and multiscale supervision. Experimental results on the MFNet and PST900 datasets demonstrate that MiLNet outperforms state-of-the-art methods in terms of mIoU. As a limitation, the model's performance under few-sample conditions could be improved further. The code and results of our method are available at https://github.com/Jinfu-pku/MiLNet.

Network feedback, representing the impact of congestion control actions on the network, is crucial for evaluating the advantages of the actions taken. Previous reinforcement learning (RL)-based congestion control algorithms use average performances over fixed periods to measure network feedback, which fails to accurately capture the impact of an action and leads to performance degradation. In this paper, we propose Zygos, which accurately estimates network feedback. This accurate feedback can benefit other RL-based congestion control algorithms. Zygos contains a distribution-based reward correction mechanism that leverages a RL model to generate relevance distributions for the sequence rewards of each state-action pair, and then aggregates the rewards by weighted average. Zygos also adopts metagradient RL to capture network feedback offset patterns, thereby updating the relevance generation model during the training of the congestion control algorithm. Experiments show that the RL congestion control method using Zygos achieves an average 20–30% improvement in throughput and 20% decrease in delay compared to the original method, highlighting substantial enhancements in RL-based congestion control algorithms.