Current multi-modal sensor trackers mainly rely on visual cues for target tracking. However, in challenging scenarios, the visual information acquired by multi-modal sensors has limited descriptive ability when the target state undergoes significant changes, which may lead to unsatisfactory tracker performance. In this work, we propose TVTracker, a two-stage tracking framework. It leverages semantic information of the target state to enhance visual cues, enabling effective RGB-T tracking. The first stage is the generation of target text descriptions. Utilizing the Bootstrapping Language-Image Pre-training (BLIP) model, we generate target textual descriptions that match the images in the dataset. In the second stage, text-guided visual fusion is performed for target tracking. Target textual and visual features are extracted separately using the text and visual branches. Then the target textual features are integrated with the visual features to localize the target position and predict the target bounding box. In the text branch, we design the Target Text Adaptation Enhancement (TTAE) module to mitigate the interference of low-quality target textual features on visual features. In the visual branch, we develop the multi-modal visual information prompters, which include the Multi-modal Visual Shared Information Prompter (MVSIP) and the Multi-modal Visual Shared and Complementary Information Prompter (MVSCIP), to facilitate learning of multi-modal shared or complementary visual prompts. Experiments on the LasHeR, RGBT210, RGBT234, and VTUAV datasets demonstrate the effectiveness of TVTracker.

In the Vision-and-Language Navigation (VLN) task, an agent must comprehend natural language instructions and execute precise navigation in complex environments. While significant progress has been made in the VLN field, the limited availability of navigation data hinders existing methods from fully learning the commonsense relationships between rooms and landmarks, which are crucial for environmental understanding and successful navigation. To address this issue, this work proposes a Visual and Textual Commonsense-Enhanced Layout Learning Model (ViTeC). We leverage the open-world knowledge embedded in large models by utilizing ChatGPT and BLIP-2 to provide commonsense information about environments. Specifically, BLIP-2 analyzes the room type corresponding to each panoramic image, while ChatGPT infers and provides knowledge about the most common landmarks within each room type. Moreover, to compensate for the agent’s lack of commonsense at the visual level, we employ Stable Diffusion to generate commonsense-based visual images, enhancing the agent’s visual perception. To ensure the agent effectively learns commonsense about the environment, we designed a Text Commonsense Layout Learning Module and a Visual Commonsense Layout Learning Module. These modules help the agent acquire environmental commonsense from both linguistic and visual perspectives, enabling it to utilize commonsense information effectively during navigation, thereby improving its environmental understanding and reasoning capabilities. Experimental results demonstrate that ViTeC achieves strong performance on REVERIE, R2R, and SOON datasets, exhibiting good generalization ability in complex environments. This validates the effectiveness of ViTeC in enhancing the agent’s environmental understanding and navigation capabilities.

Most existing Vision Transformer-based frameworks for weakly supervised semantic segmentation utilize class activation maps to generate pseudo masks. Although it mitigates the class-agnostic issue, this approach still suffers from misclassification and noise in segmentation results. To overcome these limitations, we propose an attention-based framework named Cross-block Sparse Class Token Contrast (CB-SCTC), which incorporates Dynamic Sparse Attention module (DSA) and Cross-block Class Token Contrast scheme (CB-CTC). Specifically, the proposed Cross-block Class Token Contrast scheme forces diversity between the final class tokens by learning from the lower similarity of the class tokens in the relatively shallower blocks. Moreover, the Dynamic Sparse Attention module is designed to post-process the output from the softmax function in the attention mechanism to reduce noise. Extensive experiments prove the proposed framework is a valid alternative to class activation maps. Our framework demonstrates competitive mIoU scores on the PASCAL VOC 2012(val:75.5%, test:75.2%) and MS COCO 2014 dataset(val:46.9%). Our code is available athttps://github.com/Jingfeng-Tang/CB-SCTC.

In order to solve the problems of retention time delay caused by the direct prediction for users' off-grid state and insufficient information mining caused by the too rough granularity of the monthly statistical features in the past, this paper presents the approach that applies serialized data analysis method to users' records serialized with Day as its granularity. Users' data would be sampled and analyzed by using the sliding windows. Then the churn incline among the users would be predicted in advance by collecting and sorting out their antecedent behavior occurring before their churn. The related experiments were conducted with real mobile communication records of social users and reached 98.22% recall and 0.98 F1(F1-measure) when it reached its highest level. Along with comparing with the traditional modeling which takes month as the granularity for its statistical item, the timeliness and predicting competency of the method presented in this article has been verified both theoretically and by the result of the test.