This paper presents a unified framework for continual learning in Audio-Visual Question Answering (AVQACL), designed to enhance fine-grained scene understanding and spatio-temporal reasoning in dynamic multimodal task environments. To simulate realistic incremental learning scenarios, we construct two large-scale benchmark datasets, Split-AVQA and Split-MUSIC-AVQA, by reorganizing existing AVQA corpora into sequential tasks. Empirical results show that conventional models suffer from severe performance degradation and catastrophic forgetting when learning across audio, visual, and textual modalities in a continual setup. To address these challenges, we propose a novel approach that integrates four key modules: (1) Question-Guided Cross-modal Information Fusion (QCIF), which dynamically extracts task-relevant multimodal features via question-aware attention; (2) Task-specific Knowledge Distillation with Spatial-Temporal Feature Constraints (TKD-STFC), which preserves semantic output behavior and internal reasoning trajectories across tasks; (3) Question Semantic Consistency Constraint (QSCC), which regularizes evolving question representations to maintain linguistic stability. (4) Dual-Strategy Exemplar Selection (DSES), a memory-efficient replay strategy that jointly maximizes sample informativeness and diversity. All components are theoretically grounded, and formal analysis is provided to ensure modality alignment, spatial-temporal coherence, and exemplar selection reliability. Extensive experiments on both datasets demonstrate that our method consistently outperforms prior state-of-the-art AVQACL baselines in terms of accuracy, retention, and robustness.

Textual backdoor attacks present a substantial security risk to Large Language Models (LLM). It embeds carefully chosen triggers into a victim model at the training stage and makes the model erroneously predict inputs containing the same triggers as a certain class. Prior backdoor defense methods primarily target special-token-based triggers, leaving syntax-based triggers insufficiently addressed. To fill this gap, this paper proposes a novel defense algorithm that effectively counters syntax-based as well as special-token-based backdoor attacks. The algorithm replaces semantically meaningful words in sentences with entirely different ones but preserves the syntactic templates or special tokens, and then compares the predicted labels before and after the substitution to determine whether a sentence contains triggers. Experimental results confirm the algorithm’s performance against these two types of triggers, offering a comprehensive defense strategy for model integrity.

Graph representation learning on vast datasets, like web data, has made significant strides. However, the associated computational and storage overheads raise concerns. In sight of this, Graph condensation (GCond) has been introduced to distill these large real datasets into a more concise yet information-rich synthetic graph. Despite acceleration efforts, existing GCond methods mainly grapple with efficiency, especially on expansive web data graphs. Hence, in this work, we pinpoint two major inefficiencies of current paradigms: (1) the concurrent updating of a vast parameter set, and (2) pronounced parameter redundancy. To counteract these two limitations correspondingly, we first (1) employ the Mean-Field variational approximation for convergence acceleration, and then (2) propose the objective of Gradient Information Bottleneck (GDIB) to prune redundancy. By incorporating the leading explanation techniques (e.g., GNNExplainer and GSAT) to instantiate the GDIB, our EXGC, the Efficient and eXplainable Graph Condensation method is proposed, which can markedly boost efficiency and inject explainability. Our extensive evaluations across eight datasets underscore EXGC's superiority and relevance. Code is available at https://github.com/MangoKiller/EXGC.

Collaboration can amalgamate diverse ideas, styles, and visual elements, fostering creativity and innovation among different designers. In collaborative design, sketches play a pivotal role as a means of expressing design creativity. However, designers often tend to not openly share these meticulously crafted sketches. This phenomenon of data island in the design area hinders its digital transformation under the third wave of AI. In this paper, we introduce a Federated Generative Artificial Intelligence Clothing system, namely StyleWe, employing federated learning to aid in sketch design. StyleWe is committed to establishing an ecosystem wherein designers can exchange sketch styles among themselves. Through StyleWe, designers can generate sketches that incorporate various designers' styles from their peers, drawing inspiration from collaboration without the need for data disclosure or upload. Extensive performance evaluations and user studies indicate that our StyleWe system can produce multi-styled sketches of comparable quality to human-designed ones while significantly enhancing efficiency compared to hand-drawn sketches.

The training and inference of Graph Neural Networks (GNNs) are costly when scaling up to large-scale graphs. Graph Lottery Ticket (GLT) has presented the first attempt to accelerate GNN inference on large-scale graphs by jointly pruning the graph structure and the model weights. Though promising, GLT encounters robustness and generalization issues when deployed in real-world scenarios, which are also long-standing and critical problems in deep learning ideology. In real-world scenarios, the distribution of unseen test data is typically diverse. We attribute the failures on out-of-distribution (OOD) data to the incapability of discerning causal patterns, which remain stable amidst distribution shifts. In traditional spase graph learning, the model performance deteriorates dramatically as the graph/network sparsity exceeds a certain high level. Worse still, the pruned GNNs are hard to generalize to unseen graph data due to limited training set at hand. To tackle these issues, we propose the Resilient Graph Lottery Ticket (RGLT) to find more robust and generalizable GLT in GNNs. Concretely, we reactivate a fraction of weights/edges by instantaneous gradient information at each pruning point. After sufficient pruning, we conduct environmental interventions to extrapolate potential test distribution. Finally, we perform last several rounds of model averages to further improve generalization. We provide multiple examples and theoretical analyses that underpin the universality and reliability of our proposal. Further, RGLT has been experimentally verified across various independent identically distributed (IID) and out-of-distribution (OOD) graph benchmarks.

The “mind-controlling” capability has always been in mankind's fantasy. With the recent advancements in electroencephalograph (EEG) techniques, brain-computer interface (BCI) researchers have explored some solutions to allow individuals to perform various tasks using their minds. However, the commercial off-the-shelf devices to run accurate EEG signal collection are usually expensive and the comparably cheaper devices can only present coarse results, which prevents the practical application of these devices in domestic services. To tackle this challenge, we propose and develop an end-to-end solution that enables fine brain-robot interaction (BRI) through embedded learning of coarse EEG signals from low-cost devices, namely PerBCI, so that people having difficulty moving, such as the elderly, can mind command and control a robot to perform some basic household tasks. Our contributions are three folds: 1) We present a stacked long short-term memory (BiLSTM) structure, along with specific pre-processing techniques to handle the time-dependency of EEG signals and their classification. 2) We propose a personalized design to adaptively capture multiple features and achieve accurate recognition of individual EEG signals by enhancing the signal interpretation of BiLSTM with an attention mechanism. 3) We develop a low-cost, real-time and end-to-end BRI system that can run our PerBCI models and algorithms in the embedded robot platform to perform more than one type of domestic task based on the users' EEG signal inputs. Our real-world experiments with elderly participants of diverse backgrounds in a home setting and system comparison with other approaches show that the proposed end-to-end solution with low cost can achieve satisfactory run-time speed, accuracy and energy-efficiency.

Recent advancements in EEG-based BCI technologies have been explored to assist individuals in executing brain-to-robot tasks. In the future, using BCI systems in both personal and professional domains is anticipated in widespread adoption. One promising application is facing home life, to enable people to use commercial EEG equipment to implement daily tasks. However, current BCI studies mainly focus on single-brain-to-single-robot interaction, which has limitations in representing diverse human intentions. A generalized BCI system with multiple EEG devices allows users to more precisely or collaboratively control robots. Therefore, it is imperative to extend the BCI techniques to future collaboration scenarios. In this paper, we present a new system, NeuroBCI, for multi-brain-to-multi-robot interaction through the integration of sensing, computing, communication, and control. To improve sensing efficiency, NeuroBCI employs a sparse attention mechanism to extract joint features from heterogeneous EEG data. Parallel computation and transmission for multi-user multi-task scenarios are handled by semantic autoencoder and autodecoder communications. A code map is designed to ensure concurrent control and model compression methods are used on both transmitter and receiver sides. Our experiments in comparison with state-of-the-art works show the superior performance of NeuroBCI on sensing, computing, communication, and control as a holistic system.

Recent years have witnessed significant developments in point cloud processing, including classification and segmentation. However, supervised learning approaches need a lot of well-labeled data for training, and annotation is labor-and time-intensive. Self-supervised learning, on the other hand, uses unlabeled data, and pretrains a back-bone with a pretext task to extract latent representations to be used with the downstream tasks. Compared to 2D images, self-supervised learning of 3D point clouds is under-explored. Existing models, for self-supervised learning of 3D point clouds, rely on a large number of data samples, and require significant amount of computational re-sources and training time. To address this issue, we propose a novel contrastive learning approach, referred to as To ThePoint. Different from traditional contrastive learning methods, which maximize agreement between features obtained from a pair of point clouds formed only with different types of augmentation, ToThePoint also maximizes the agreement between the permutation invariant features and features discarded after max pooling. We first perform self-supervised learning on the ShapeNet dataset, and then evaluate the performance of the network on different downstream tasks. In the downstream task experiments, performed on the ModelNet40, ModelNet40C, ScanobjectNN and ShapeNet-Part datasets, our proposed ToThe-Point achieves competitive, if not better results compared to the state-of-the-art baselines, and does so with significantly less training time (200 times faster than baselines).

This letter focuses on a modified complex-valued Markov random field (CMRF) modeling filter to improve the performance in the filtering of the synthetic aperture radar interferometric phase. In the reference CMRF modeling phase map filter, the CMRF model was employed to update residues and their neighbors. By this, the residues were reduced, and phase jumps were preserved simultaneously. However, residue reduction was still insufficient. Furthermore, incorrect model parameter estimation leads to wrong filtering in some blocks. To solve the two aforementioned problems, we propose a modified CMRF modeling filter, where the original interferogram is divided into overlapped blocks. In addition, the adaptive weighted neighbor values are used to estimate the CMRF model parameters. Both simulated and real data experiments are performed to validate this method.

In this letter, an approach based on spectral diversity and conventional Interferometric Synthetic Aperture Radar (InSAR) for Digital Elevation Model (DEM) reconstruction is proposed. Conventional InSAR technique often fails to correctly reconstruct the DEM due to the presence of discontinuities and/or interferometric noise. With the assistance of the spectral diversity method, the ill-posed phase-unwrapping can be regularized. Thus the capabilities of the conventional InSAR technique for DEM reconstruction can be notably improved. Comparing to the multi-channel InSAR technique, the proposed method requires less SAR images. Comparing to the multifrequency technique, the performance of the proposed method depends on the perpendicular or effective baseline and the ratio of the radar's carry frequency to the bandwidth between the central frequencies of the two sub-looks; while the multifrequency technique depends more on the range bandwidth. Thus, the proposed method is more flexible than the multifrequency technique. The effectiveness of the proposed algorithm is validated by experimental results using real data.

In this paper, an adaptive patch-based filter for Synthetic aperture Radar (SAR) Interferometric phase in the Fourier transform complex domain is proposed. The proposed filter fully utilizes the properties of locally planar terrain phase noise model so that the noise can be suppressed according to the local level of noise. Moreover, the denoising performance of the proposed filter is further improved by an adaptive selection strategy for patch size. The experimental results show that the proposed method delivers state-of-the-art denoising performance in terms of objective criteria.