Traditional vision-based material perception methods often experience substantial performance degradation under visually impaired conditions, thereby motivating the shift toward non-visual multimodal material perception. Despite this, existing approaches frequently perform naive fusion of multimodal inputs, overlooking key challenges such as modality-specific noise, missing modalities common in real-world scenarios, and the dynamically varying importance of each modality depending on the task. These limitations lead to suboptimal performance across several benchmark tasks. In this paper, we propose a robust multimodal fusion framework, TouchFormer. Specifically, we employ a Modality-Adaptive Gating (MAG) mechanism and intra- and inter-modality attention mechanisms to adaptively integrate cross-modal features, enhancing model robustness. Additionally, we introduce a Cross-Instance Embedding Regularization(CER) strategy, which significantly improves classification accuracy in fine-grained subcategory material recognition tasks. Experimental results demonstrate that, compared to existing non-visual methods, the proposed TouchFormer framework achieves classification accuracy improvements of 2.48% and 6.83% on SSMC and USMC tasks, respectively. Furthermore, real-world robotic experiments validate TouchFormer's effectiveness in enabling robots to better perceive and interpret their environment, paving the way for its deployment in safety-critical applications such as emergency response and industrial automation.

Tracking methods for measuring the similarity between the template and search region have achieved great success in recent years. Although many researchers have made efforts to introduce template annotations into network, inductive bias for trackers is unavoidable due to the inherent disadvantage of box representation. In this work, a novel tracking framework is proposed to eliminate the misguidance of biased prior, based on which, a target-aware Transformer tracker is designed. We use the template annotation as a predicted item in supervised learning, train our model to estimate the same target in template and search frame simultaneously, so that the tracker can learn the target-awareness both in the past and present frame. Our method can be assembled on the vast majority of Transformer-based networks. Sufficient experiments on six datasets verify the correctness of the proposed model. Without the bells and whistles, our tracker achieves the state-of-the-art performance on multiple benchmarks.

Compared with the current booming development of siamese tracking network, online optimization methods for tracking models still update parameters or features in pulses, which is non-real-time and on whole image level. In the past year, similarity measurement components derived from Transformer equiped on Siamese networks have obtained excellent performance in visual tracking task. Leveraging its element-wise attention mechanism, we implement a real-time feature update approach on coarse pixel level. We first construct a classification branch for quality control; and to further reduce the feature amount in online update process, we apply an incremental clustering method to minimize the repetitive contribution of similar features. The proposed method is evaluated on multiple datasets including OTB2015, NfS and GOT-10k. It exceeds the baseline methods on all 3 datasets and achieves competitive performance against the state-of-the-art networks.

Analysis on changes of image features is an effective means to leverage multi-dimensional information in motion segmentation. Methods without considering temporal perspective are agnostic of motion state, and inherently unsuitable for motion detection. However, the effect of existing spatio-temporal approaches is lagging behind that of spatial methods by a margin. To make better use of temporal information, this paper tackles the task of moving object segmentation by constructing a Change-aware Siamese neural network(ChaSiam) to detect relative foreground and changes. Further, a reference frame update strategy is attached to our network for overcoming the weakness of spatio-temporal approaches in cases with camera ego-motion. Extensive experiments show that our proposed model outperforms previous state-of-the-art spatio-temporal methods on Change Detection dataset, and compared with spatial methods our model has similar performance with better generalization.