Although Contrastive Language-Image Pretraining (CLIP) exhibits strong performance across diverse vision tasks, its application to person representation learning faces two critical challenges: (i) the scarcity of large-scale annotated vision-language data focused on person-centric images, and (ii) the inherent limitations of global contrastive learning, which struggles to maintain discriminative local features crucial for fine-grained matching while remaining vulnerable to noisy text tokens.This work advances CLIP for person representation learning through synergistic improvements in data curation and model architecture.First, we develop a noise-resistant data construction pipeline that leverages the in-context learning capabilities of MLLMs to automatically filter and caption web-sourced images.This yields WebPerson, a large-scale dataset of 5M high-quality person-centric image-text pairs.Second, we introduce the GA-DMS (Gradient-Attention Guided Dual-Masking Synergetic) framework, which improves cross-modal alignment by adaptively masking noisy textual tokens based on the gradient-attention similarity score.Additionally, we incorporate masked token prediction objectives that compel the model to predict informative text tokens, enhancing fine-grained semantic representation learning.Extensive experiments show that GA-DMS achieves state-of-the-art performance across multiple benchmarks.

Advanced scene understanding is crucial for robots to navigate robustly in complex 3D environments. Recent works utilize large Vision-Language Models (VLMs) to embed semantic information into reconstructed maps, thereby creating open-vocabulary scene representations for instance-aware robot navigation. However, existing methods primarily generate point-wise feature vectors for maps, which inadequately capture the intricate scene contents necessary for navigation tasks, including holistic and relational object information. To address this limitation, we propose a novel Dual-Level Open-Vocabulary 3D (DLOV-3D) scene representation framework to improve robot navigation performance. Our framework integrates both pixel-level and image-level features into spatial scene representations, facilitating a more comprehensive understanding of the scene. By incorporating an adaptive revalidation mechanism, DLOV-3D achieves precise instance-aware navigation based on free-form queries that describe object properties such as color, shape, and relational references. Notably, when combined with Large Language Models (LLMs), DLOV-3D supports long-sequence multi-instance robot navigation guided by natural language instructions. Extensive experimental results demonstrate that DLOV-3D achieves new state-of-the-art performance in instance-aware robot navigation.

In brain-computer interface (BCI) systems, symmetric positive definite (SPD) manifold within Riemannian space has been frequently utilized to extract spatial features from electroencephalogram (EEG) signals. However, the intrinsic high dimensionality of SPD matrices introduces too much computational burden to hinder the real-time applications of such BCI, especially in handling dynamic tasks, like incremental learning. Directly reducing the dimensionality of SPD matrices with conventional dimensionality reduction (DR) methods will alter the fundamental properties of SPD matrices. Moreover, current DR methods for incremental learning always necessitate retaining old data to update their representations under new mapping. To this end, a bidirectional two-dimensional principal component analysis for SPD manifold (B2DPCA-SPD) is proposed to reduce the dimensionality of SPD matrices, in such way that the reduced matrices remain on SPD manifold. Afterwards, the B2DPCA-SPD is extended to adapt to incremental learning tasks without saving old data. The incremental B2DPCA-SPD can be seamlessly integrated with the matrix-formed growing neural gas network (MF-GNG) to achieve an incremental EEG classification, where the new low-dimensional representations of the prototypes in old classifiers can be easily recalculated with the updated projection matrix. Extensive experiments are conducted on two public datasets to perform the EEG classification. The results demonstrate that our method significantly reduces computation time by 38.53% and 35.96%, and outperforms conventional methods in classification accuracy by 4.21% to 19.59%.

Current surface electromyography (sEMG)-based gesture recognition only extracts time or frequency features from raw sEMG signals, and then puts the features together to generate sample vectors, which are further used as inputs to build fixed classification models. This way may bring out two issues. First, raw sEMG signals are often acquired from multichannel electrodes. Only extracting time or frequency features will lose the spatial topology information between different channels, and cannot reflect the movement synergy of different muscles, causing relatively low recognition accuracies. Second, fixed classifiers only recognize fixed gestures, and cannot handle dynamically increasing gestures, limiting the scalabilities of classifiers in applications. To this end, we introduce a myoelectric manifold representation based on the symmetric positive definite (SPD) matrix to express the spatial synergy of multiple muscles. Then, the growing neural gas network is extended to the SPD manifold space, and uses myoelectric matrices as inputs to realize the incremental gesture recognition, maintaining the space topology with very few prototypes. Extensive experiments were conducted on two public databases (Ninapro DB2 and DB5) and a self-collection database. Experimental results showed that our method was superior to current methods, increasing recognition accuracy by 1.63%–11.89%, and can continuously grow its recognition ability online, revealing the potential in implementing myoelectric interaction systems.

Arbitrary style transfer is attracting increasing attention in the computer vision community due to its application flexibility. Existing approaches directly fuse deep style features with deep content features or adaptively normalize content features for global statistical matching. Although effective, it is prone to suffer from local unnatural outputs and artifacts owing to the lack of exploring the global contextual semantic distribution of style image features. In this article, a novel global context self-attentional network (GCSANet) is proposed to efficiently generate high-quality stylized results based on the global semantic spatial distributions of style images. First, a context modeling module is proposed to aggregate the depth features of style images into global context features. Then, channel-wise interdependencies are captured with the feature transform module. Finally, the style features are appropriately aggregated to each location of the content image. In addition, novel external contrastive losses are proposed to balance the distribution of content and style features to ensure the reasonableness of the texture patterns in the stylized images. The ablation studies validate the effectiveness of the proposed components. Various quantitative and qualitative experiments demonstrate the superiority of our method for real-time arbitrary image/video style transfer.

Side-scan sonar (SSS) is attracting increasing attention in ocean exploration for its utility and stability on autonomous underwater vehicles (AUVs). Existing SSS image recognition methods mainly employ deep neural networks (DNNs) for various tasks. However, the effectiveness of DNN-based approaches is limited in situations with zero samples. In this letter, a multifeature fusion self-attention network (MFSANet) is proposed to generate SSS images of novel categories, transforming this problem into a conventional supervised learning problem. Specifically, optical-acoustic image pairs are used as inputs to the network to synthesize pseudo-SSS images. First, the shallow and deep features of the input images are extracted by different layers of the encoder. Then, the long-range dependencies of the acoustic images are efficiently modeled with the proposed simplified self-attention module (SSAM). Finally, the acoustic features are appropriately aggregated to each position of the optical features to efficiently generate pseudo-SSS samples for training the classification network. In addition, a novel contrastive loss is proposed to optimize the cross-modal feature space distribution. Experimental results demonstrate that our method can efficiently generate high-quality pseudo-SSS samples, which improves the accuracy of zero-shot SSS image recognition.

sEMG recognition has been used extensively in prosthetic device control, human-assisting manipulators and sign language recognition, etc. However, the sEMG recognition model, trained with one subject's sEMG data, is not applicable to the other subjects, which hinders the practical application of myoelectric interfaces immensely. In this paper, a sEMG recognition method which is applicable to multi-users is proposed. Firstly, single channel sEMG is decomposed into 30 MUAPTs, which includes four steps: two-order differential filter, threshold calculation, spike detection and hierarchical clustering. Secondly, the MUAPTs are updated with the templates orthogonalization; and Deep Boltzman Machine is employed to classify the MUAPTs into five classes corresponding to the predefined five gestures. Six participants participated in this experiment to validate the effectiveness of the proposed method. Results indicated that this method can achieve a mean accuracy of 81.5%.

Electromyography (EMG) has been widely used as control commands for prosthesis, powered exoskeletons and rehabilitative robots. In this paper, an EMG-driven state space model is developed to estimate continuous joint angular displacement and velocity, demonstrated by elbow flexion/ extension. The model combines the Hill-based muscle model with the forward dynamics of joint movement, in which kinematic variables are expressed as a function of neural activation levels. EMG features including integral of absolute value and waveform length are then extracted, and two quadratic equations which associate the kinematic variables with EMG features are constructed to represent the measurement equation. The proposed model are verified by extensively experiments, where the angular movements of human elbow joint are estimated only using the EMG signals, and the estimations are compared with the IMU measurements to validate the accuracy. As a demonstration, a robotic arm is commanded to follow the human elbow movement estimated by the proposed model, which shows the possibility of EMG-based robotic assisted rehabilitation.