Zero-shot recognition has attracted increasing attention for recognizing unseen classes by aligning image visual features and class semantic features. Although various visual and semantic feature types have been explored in literature,e.g., semantic-unaware/aware visual features and attribute/name-based semantic features, most existing methods are tailored to specific bimodal feature pairs, leading to degraded performance when generalized to other feature combinations. To address this issue, we analyze the commonly-used visual and semantic features and reveal that single-modal features generally contain either redundant or deficient information, and bimodal features generally suffer from cross-modal inconsistency. Accordingly, we propose a bimodal commuting alignment network, called BCA-Net, which could accommodate various bimodal feature combinations. Specifically, an entropy-based perception module is first designed to learn bimodal prompts by strengthening discriminative visual-channel features and semantic tokens respectively. Then, a bimodal complementing module is proposed to refine the learned bimodal prompts for complementing inconsistent information across feature modalities. Extensive experimental results on 13 public datasets demonstrate that BCA-Net consistently outperforms 20 state-of-the-art methods by utilizing 4 bimodal feature combinations in most cases, highlighting its strong generalizability beyond designs tailored to specific bimodal pairs.

Image recognition is a classic and common task that has been widely applied in computer vision in the past decade. Most existing methods in literature aim to learn discriminative features for classification from labeled images; however, they generally neglect confounders that infiltrate the learned features, resulting in low discrimination performances on test images. To address this problem, we propose a Recursive Counterfactual Deconfounding model for image recognition in both closed- and open-set scenarios based on counterfactual analysis, called RCD. The proposed model consists of a factual graph and a counterfactual graph, where the relationships among image features , model predictions, and confounders are built and updated recursively to learn more discriminative features. It performs recursively so that subtler counterfactual features can be learned and progressively eliminated. Both the discriminability and generalizability of the proposed model can be improved accordingly. In addition, a negative correlation constraint is designed to alleviate the negative effects of counterfactual features during model training. Extensive experimental results in both the closed-set and open-set image recognition tasks demonstrate the effectiveness of the proposed method. Particularly under the hard testing mode of the three public datasets (Aircraft, CUB and ImageNet), the proposed method obtains a higher AUROC (Area Under the Receiver Operating Characteristic curves) by 0.7%–3.5% and a higher OSCR (Open-Set Classification Rate) by 0.2%–4.0%.

Generative models have attracted much attention for handling the generalized zero-shot learning (GZSL) task recently. Most of the existing generative GZSL models are trained for visual feature synthesis by utilizing the unique semantic feature of each object class as input but its kaleidoscopic real visual features as supervisions. However, since the real visual features are inevitably infiltrated by some class-irrelevant information, the trained generative models could not guarantee the discriminability of their synthesized visual features. In this paper, we firstly provide an empirical analysis on this problem, finding that among the elements of the real visual features, some elements contain more class-irrelevant information than the others, resulting in ambiguous visual feature synthesis. Then according to this finding, we propose a self-assembled generative GZSL framework, where both the real and synthesized visual features are re-assembled by identifying and updating the class-irrelevant elements in a self-learning manner, called SaG. Moreover, an element-affinity regularizer is explored for constraining the affinity among different elements, so that the synthesized visual features under the SaG framework approach the updated feature elements. In principle, different generative GZSL models could be seamlessly embedded into the SaG framework, resulting in different GZSL methods. Extensive experimental results demonstrate that the derived methods, by embedding three baseline generative GZSL models into SaG respectively, could boost the performances of their baselines significantly, and one of the derived methods outperforms 20 state-of-the-art GZSL methods in most cases.

Recently, deep neural networks have been extensively used to extract features from EEG data for epileptic seizure prediction in the epilepsy diagnosis community. Many existing works in literature either use the ultimate-layer feature or aggregate multi-layer features via straightforward concatenation or element-wise addition, but they do not pay a special attention to the contextual consistency between these features as well as the involved noise in these features. To address the above problem, we propose a Noise-aware epileptic seizure prediction network via Self-attention Feature Alignment, called NSFA-Net. The NSFA-Net consists of two modules: a self-attention backbone module to extract multi-layer features from the input EEG data, and a time-frequency feature alignment module to align these features for maintaining the contextual consistency. In addition, during the training process, a noise-aware regularizer is introduced to alleviate the negative influence of noise that is generally inevitable in EEG data. The average sensitivities of the proposed method on the CHB-MIT and Kaggle datasets are 98.68% and 93.57% respectively, and the average false prediction rates are 0.038/h and 0.060/h respectively. These experimental results show the superiority of the proposed method to some state-of-the-art methods.

Generalized zero-shot learning (GZSL) is a challenging topic in both computer vision and machine learning. Recently, generative models (e.g., GAN and VAE) have attracted much attention for handling the GZSL task, however, they are sometimes prone to either model collapse or ambiguous distribution modeling. Inspired by the feature generation ability of denoising diffusion models in other visual tasks, we propose an Adaptive Conditional Denoising Diffusion Model to synthesize unseen-class visual features for GZSL on condition of a set of semantic features in this paper, called AC-DDM. Unlike traditional denoising diffusion models whose reverse process has both a fixed time interval and a fixed number of total denoising time steps, the proposed AC-DDM has a learnable distribution-constrained predictor which could adaptively learn the time interval and the number of total denoising time steps for each unseen class, so that it could synthesize more discriminative features for sample classification. In order to improve the discrimination ability of the synthesized visual features further, we also explore a hybrid affinity regularizer under the proposed AC-DDM, which forces the differences among the affinity matrices of the real and synthesized visual features to be small. Extensive experimental results on four public benchmark datasets demonstrate the superiority of the proposed model over 20 state-of-the-art models in both the ZSL and GZSL tasks.

SAR (Synthetic Aperture Radar) image semantic segmentation has attracted increasing attention in the remote sensing community recently, due to SAR’s all-time and all-weather imaging capability. However, SAR images are generally more difficult to be segmented than their EO (Electro-Optical) counterparts, since speckle noises and layovers are inevitably involved in SAR images. On the other hand, EO images could only be obtained under cloud-free conditions, which limits their applications. To this end, this letter investigates how to introduce EO features to assist the training of a SAR-segmentation model so that the model could segment SAR images without their EO counterparts in application, and proposes a distilled heterogeneous feature alignment network (DHFA-Net), where a SAR-segmentation student model learns and aligns the features from a pre-trained EO-segmentation teacher model. In the proposed DHFA-Net, both the student and teacher models employ an identical architecture but different parameter configurations, and a heterogeneous feature distillation module is explored for transferring latent EO features from the teacher model to the student model through heterogeneous feature distillation and then supervising the training of the SAR-segmentation model. Moreover, a heterogeneous feature alignment module is designed to aggregate multi-scale features for segmentation by feature alignment approach in each of the student and teacher models. By enabling the multi-scale heterogeneous feature aggregation, the SAR segmentation performance could be boosted. Experimental results on two public datasets demonstrate the superiority of the proposed DHFA-Net.

Finding justifications for an entailment is one of the major missions in the field of ontology research. Recent advances on finding justifications w.r.t. the light-weight description logics focused on encoding this problem into a propositional formula, and using SAT-based techniques to enumerate all MUSes (minimally unsatisfiable subformulas). It's necessary to import more optimized techniques into finding justifications as emergence of large-scale real-world ontologies. In this paper, we propose a new strategy which introduce local search(in short, LS) technique to compute the approximating core before extracting an exact MUS. Although it is based on a heuristic and LS, such technique is complete in the sense that it always delivers a MUS for any unsatisfiable SAT instance. Our method will find the justifications for large-scale ontologies more effectively.