Multimodal keyphrase generation (MKP) aims to extract a concise set of keyphrases that capture the essential meaning of paired image–text inputs, enabling structured understanding, indexing, and retrieval of multimedia data across the web and social platforms. Success in this task demands effectively bridging the semantic gap between heterogeneous modalities. While multimodal large language models (MLLMs) achieve superior cross-modal understanding by leveraging massive pretraining on image-text corpora, we observe that they often struggle with modality bias and fine-grained intra-modal feature extraction. This oversight leads to a lack of robustness in real-world scenarios where multimedia data is noisy, along with incomplete or misaligned modalities. To address this problem, we propose AimKP, a novel framework that explicitly reinforces intra-modal semantic learning in MLLMs while preserving cross-modal alignment. AimKP incorporates two core innovations: (i) Progressive Modality Masking, which forces fine-grained feature extraction from corrupted inputs by progressively masking modality information during training; (ii) Gradient-based Filtering, that identifies and discards noisy samples, preventing them from corrupting the model’s core cross-modal learning. Extensive experiments validate AimKP’s effectiveness in multimodal keyphrase generation and its robustness across different scenarios.

Electroencephalogram (EEG) artifact removal has been investigated for decades with the goal of reconstructing the clean signals for the subsequent EEG analysis. However, existing denoising methods still have limited capabilities to handle the highly mixed artifacts and the fine-grained temporal dependency of artifact-free EEG without a priori knowledge of the artifacts. To address the challenges, this study proposes a CNN-Transformer-based dual-stage collaborative ensemble learning framework (namely CT-DCENet) in the form of three modules: 1) randomized collaboration module initially utilizes four individual learners to reveal multi-group morphological characteristics of the denoised EEG, 2) linear ensemble module integrates the outputs of four individual learners via weighted linear combination to preliminarily estimate the denoised EEG, 3) information complementation module takes in the residual between the contaminated EEG and the above estimated EEG, and critically applies CNN-Transformer-based feature extractor and denoising head to learn the detailed characteristics of the denoised EEG. CT-DCENet is conducted in a dual-stage training manner to derive the morphological characteristics & the detailed characteristics of the artifact-free EEG successively. The experimental results on the public EEG datasets indicate that 1) CT-DCENet significantly outperforms the state-of-the-art counterparts (e.g., DuoCL, GCTNet) under the conditions of various artifacts and noise intensities, where the increases of SNR & PCC are 0.79 dB, 0.6% and the decrease of RRMSE is 1.9% for the removal of EMG, ECG, EOG mixed artifacts, 2) the reconstructed EEG by CT-DCENet can well fit the clean EEG with a low error achieved, especially for the peak amplitude, the high-frequency area and the boundary area of the EEG waveform, providing promising EEG data for the downstream task-oriented EEG analysis.

Analysis of neuroimaging data (e.g., Magnetic Resonance Imaging, structural and functional MRI) plays an important role in monitoring brain dynamics and probing brain structures. Neuroimaging data are multi-featured and non-linear by nature, and it is a natural way to organise these data as tensors prior to performing automated analyses such as discrimination of neurological disorders like Parkinson's Disease (PD) and Attention Deficit and Hyperactivity Disorder (ADHD). However, the existing approaches are often subject to performance bottlenecks (e.g., conventional feature extraction and deep learning based feature construction), as these can lose the structural information that correlates multiple data dimensions or/and demands excessive empirical and application-specific settings. This study proposes a Deep Factor Learning model on a Hilbert Basis tensor (namely, HB-DFL) to automatically derive latent low-dimensional and concise factors of tensors. This is achieved through the application of multiple Convolutional Neural Networks (CNNs) in a non-linear manner along all possible dimensions with no assumed a priori knowledge. HB-DFL leverages the Hilbert basis tensor to enhance the stability of the solution by regularizing the core tensor to allow any component in a certain domain to interact with any component in the other dimensions. The final multi-domain features are handled through another multi-branch CNN to achieve reliable classification, exemplified here using MRI discrimination as a typical case. A case study of MRI discrimination has been performed on public MRI datasets for discrimination of PD and ADHD. Results indicate that 1) HB-DFL outperforms the counterparts in terms of FIT, mSIR and stability (mSC and umSC) of factor learning; 2) HB-DFL identifies PD and ADHD with an accuracy significantly higher than state-of-the-art methods do. Overall, HB-DFL has significant potentials for neuroimaging data analysis applications with its stability of automatic construction of structural features.

Brain functional connectivity has been widely explored to reveal the functional interaction dynamics between the brain regions. However, conventional connectivity measures rely on deterministic models demanding application-specific empirical analysis, while deep learning approaches focus on finding discriminative features for state classification, having limited capability to capture the interpretable connectivity characteristics. To address the challenges, this study proposes a self-supervised triplet network with depth-wise attention (TripletNet-DA) to generate the functional connectivity: 1) TripletNet-DA firstly utilizes channel-wise transformations for temporal data augmentation, where the correlated & uncorrelated sample pairs are constructed for self-supervised training, 2) Channel encoder is designed with a convolution network to extract the deep features, while similarity estimator is employed to generate the similarity pairs and the functional connectivity representations, 3) TripletNet-DA applies Triplet loss with anchor-negative similarity penalty for model training, where the similarities of uncorrelated sample pairs are minimized to enhance model's learning capability. Experimental results on pathological EEG datasets (Autism Spectrum Disorder, Major Depressive Disorder) indicate that 1) TripletNet-DA demonstrates superiority in both ASD discrimination and MDD classification than the state-of-the-art counterparts, where the connectivity features in beta & gamma bands have respectively achieved the accuracy of 97.05%, 98.32% for ASD discrimination, 89.88%, 91.80% for MDD classification in the eyes-closed condition and 90.90%, 92.26% in the eyes-open condition, 2) TripletNet-DA enables to uncover significant differences of functional connectivity between ASD EEG and TD ones, and the prominent connectivity links are in accordance with the empirical findings, thus providing potential biomarkers for clinical ASD analysis.

Electroencephalogram (EEG) excels in portraying rapid neural dynamics at the level of milliseconds, but its spatial resolution has often been lagging behind the increasing demands in neuroscience research or subject to limitations imposed by emerging neuroengineering scenarios, especially those centering on consumer EEG devices. Current superresolution (SR) methods generally do not suffice in the reconstruction of high-resolution (HR) EEG as it remains a grand challenge to properly handle the connection relationship amongst EEG electrodes (channels) and the intensive individuality of subjects. This study proposes a deep EEG SR framework correlating brain structural and functional connectivities (Deep-EEGSR), which consists of a compact convolutional network and an auxiliary fully connected network for filter generation (FGN). Deep-EEGSR applies graph convolution adapting to the structural connectivity amongst EEG channels when coding SR EEG. Sample-specific dynamic convolution is designed with filter parameters adjusted by FGN conforming to functional connectivity of intensive subject individuality. Overall, Deep-EEGSR operates on low-resolution (LR) EEG and reconstructs the corresponding HR acquisitions through an end-to-end SR course. The experimental results on three EEG datasets (autism spectrum disorder, emotion, and motor imagery) indicate that: 1) Deep-EEGSR significantly outperforms the state-of-the-art counterparts with normalized mean squared error (NMSE) decreased by 1%-6% and the improvement of signal-to-noise ratio (SNR) up to 1.2 dB and 2) the SR EEG manifests superiority to the LR alternative in ASD discrimination and spatial localization of typical ASD EEG characteristics, and this superiority even increases with the scale of SR.

Artifact removal has been an open critical issue for decades in tasks centering on EEG analysis. Recent deep learning methods mark a leap forward from the conventional signal processing routines; however, those in general still suffer from insufficient capabilities 1) to capture potential temporal dependencies embedded in EEG and 2) to adapt to scenarios without a priori knowledge of artifacts. This study proposes an approach (namely DuoCL) to deep artifact removal with a dual-scale CNN (Convolutional Neural Network)-LSTM (Long Short-Term Memory) model, operating on the raw EEG in three phases: 1) Morphological Feature Extraction, a dual-branch CNN utilizes convolution kernels of two different scales to learn morphological features (individual sample); 2) Feature Reinforcement, the dual-scale features are then reinforced with temporal dependencies (inter-sample) captured by LSTM; and 3) EEG Reconstruction, the resulting feature vectors are finally aggregated to reconstruct the artifact-free EEG via a terminal fully connected layer. Extensive experiments have been performed to compare DuoCL to six state-of-the-art counterparts (e.g., 1D-ResCNN and NovelCNN). DuoCL can reconstruct more accurate waveforms and achieve the highest ${\mathsf{SNR}}$ & correlation (${\mathsf{CC}}$) as well as the lowest error (${\mathsf{RRMSE}}_{\mathsf{t}}$ & ${\mathsf{RRMSE}}_{\mathsf{f}}$). In particular, DuoCL holds potentials in providing a high-quality removal of unknown and hybrid artifacts.

Multivariate time series data (Mv-TSD) portray the evolving processes of the system(s) under examination in a “multi-view” manner. Factorization methods are salient for Mv-TSD analysis with the potentials of structural feature construction correlating various data attributes. However, research challenges remain in the derivation of factors due to highly scattered data distribution of Mv-TSD and intensive interferences/outliers embedded in the source data. The proposed Enhanced Bayesian Factorization approach (Enhanced-BF) addresses the challenges in three phases: (1) variant scale partitioning applies to Mv-TSD according to degree of amplitude and obtains the blocks of variant scales; (2) hierarchical Bayesian model for tensor factorization automatically derives the factors of each block with interferences suppressed; (3) Bayesian unification model merges those block factors to construct the final structural features.Enhanced-BFhas been evaluated using a case study of brain data engineering with multivariate electroencephalogram (EEG). Experimental results indicate that the proposed method manifests robustness to the interferences and outperforms the counterparts in terms of operation efficiency and error when factorizing EEG tensor. Besides,Enhanced-BFexcels in factorization-based analysis of ongoing autism spectrum disorder (ASD) EEG: 3 times speed-up in factorization and$87.35\%$accuracy in ASD discrimination. The latent factors (“biomarkers”) can distinctly interpret the typical EEG characteristics of ASD subjects.

Extracting the latent factors of big time series data is an important means to examine the dynamic complex systems under observation. These low-dimensional and “small” representations reveal the key insights to the overall mechanisms, which can otherwise be obscured by the notoriously high dimensionality and scale of big data as well as the enormously complicated interdependencies amongst data elements. However, grand challenges still remain: (1) to incrementally derive the multi-mode factors of the augmenting big data and (2) to achieve this goal under the circumstance of insufficient a priori knowledge. This study develops an incrementally parallel factorization solution (namely I-PARAFAC) for huge augmenting tensors (multi-way arrays) consisting of three phases over a cutting-edge GPU cluster: in the “giant-step” phase, a variational Bayesian inference (VBI) model estimates the distribution of the close neighborhood of each factor in a high confidence level without the need for a priori knowledge of the tensor or problem domain; in the “baby-step” phase, a massively parallel Fast-HALS algorithm (namely G-HALS) has been developed to derive the accurate subfactors of each subtensor on the basis of the initial factors; in the final fusion phase, I-PARAFAC fuses the known factors of the original tensor and those accurate subfactors of the “increment” to achieve the final full factors. Experimental results indicate that: (1) the VBI model enables a blind factor approximation, where the distribution of the close neighborhood of each final factor can be quickly derived (10 iterations for the test case). As a result, the model of a low time complexity significantly accelerates the derivation of the final accurate factors and lowers the risks of errors; (2) I-PARAFAC significantly outperforms even the latest high performance counterpart when handling augmenting tensors, e.g., the increased overhead is only proportional to the increment while the latter has to repeatedly factorize the whole tensor, and the overhead in fusing subfactors is always minimal; (3) I-PARAFAC can factorize a huge tensor (volume up to 500 TB over 50 nodes) as a whole with the capability several magnitudes higher than conventional methods, and the runtime is in the order of 1/n to the number of compute nodes; (4) I-PARAFAC supports correct factorization-based analysis of a real 4-order EEG dataset captured from a variety of epilepsy patients. Overall, it should also be noted that counterpart methods have to derive the whole tensor from the scratch if the tensor is augmented in any dimension; as a contrast, the I-PARAFAC framework only needs to incrementally compute the full factors of the huge augmented tensor.