Recently, cross-spectral image patch matching based on feature relation learning has attracted extensive attention. However, existing methods focus on mining richer feature relations by building complex relation extraction structures. Meanwhile, performance bottlenecks have gradually emerged. To address this, we make the first attempt to explore a stable and efficient bridge between descriptor learning and metric learning, and construct a Knowledge-Guided Learning Network (KGL-Net), which achieves significant performance improvements while abandoning complex network structures. Specifically, we find that there is feature extraction consistency between metric learning based on feature difference learning and descriptor learning based on Euclidean distance. This provides the foundation for bridge building. To ensure the stability and efficiency of the constructed bridge, on the one hand, we conduct an in-depth exploration of 20 combined network architectures. On the other hand, a feature-guided loss is constructed to achieve mutual guidance of features. In addition, unlike existing methods, we consider that the feature mapping ability of the metric branch should receive more attention. Therefore, a hard negative sample mining for metric learning (HNSM-M) strategy is constructed. To the best of our knowledge, this is the first time that hard negative sample mining for metric networks has been implemented and brings significant performance gains. Extensive experimental results show that our KGL-Net achieves SOTA performance in multiple cross-spectral image patch matching datasets. Our code is available at https://github.com/YuChuang1205/KGL-Net.

The cross-view geo-localization task aims to accurately retrieve a location using images captured from different platforms, such as satellites and drones, which is particularly challenging due to a large variation in viewpoint. Current methods mainly focus on rigid strategies like partitioning or sorting local features, which may be ill-suited to accommodate the variance of viewpoint and distance scale in different camera perspectives. To address these issues, we propose a novel method called query-driven feature learning (QDFL) to query viewpoint-invariant feature vectors autonomously. Our method incorporates an adaptive query embedding unit (AQEU) and a feature fusion unit (FFU). AQEU adjusts feature map and implements a coarse query process to extract the contextual clues. FFU further refines feature map, fusing it at spatial and channel dimensions, tending to withdraw more fine-grained features. Subsequently, AQEU executes a fine query on salient landmarks in the fused feature map, enhancing the minutia descriptive power of query vectors. Additionally, we employ parameter-efficient transfer learning manner by integrating tunable adapters into the frozen pre-trained backbone, maintaining feature representation capabilities of foundation models while enabling seamless adaptation to cross-view geo-localization task. Extensive experiments show that our method achieves state-of-the-art performances on two well-known datasets, University-1652 and SUES-200. Moreover, our method exhibits an excellent generalizability compared with current state-of-the-art methods in cross-dataset experiments. The code is available at https://github.com/Shuyu-Hu/QDFL.

Infrared small target detection faces the problem that it is difficult to effectively separate the background and the target. Existing deep learning-based methods focus on edge and shape features, but ignore the richer structural differences and detailed information embedded in high-frequency components from different directions, thereby failing to fully exploit the value of high-frequency directional features in target perception. To address this limitation, we propose a multi-scale direction-aware network (MSDA-Net), which is the first attempt to integrate the high-frequency directional features of infrared small targets as domain prior knowledge into neural networks. Specifically, to fully mine the high-frequency directional features, on the one hand, a high-frequency direction injection (HFDI) module without trainable parameters is constructed to inject the high-frequency directional information of the original image into the network. On the other hand, a multi-scale direction-aware (MSDA) module is constructed, which promotes the full extraction of local relations at different scales and the full perception of key features in different directions. In addition, considering the characteristics of infrared small targets, we construct a feature aggregation (FA) structure to address target disappearance in high-level feature maps, and a feature calibration fusion (FCF) module to alleviate feature bias during cross-layer feature fusion. Extensive experimental results show that our MSDA-Net achieves state-of-the-art (SOTA) results on multiple public datasets. The code can be available at https://github.com/YuChuang1205/MSDA-Net.

Open-set Classification (OSC) is a critical requirement for safely deploying machine learning models in the open world, which aims to classify samples from known classes and reject samples from out-of-distribution (OOD). Existing methods exploit the feature space of trained network and attempt at estimating the uncertainty in the predictions. However, softmax-based neural networks are found to be overly confident in their predictions even on data they have never seen before and the immense diversity of the OOD examples also makes such methods fragile. To this end, we follow the idea of estimating the underlying density of the training data to decide whether a given input is close to the in-distribution (IND) data and adopt Energy-based models (EBMs) as density estimators. A novel energy-based generative open-set node classification method, \textit{EGonc}, is proposed to achieve open-set graph learning. Specifically, we generate substitute unknowns to mimic the distribution of real open-set samples firstly, based on the information of graph structures. Then, an additional energy logit representing the virtual OOD class is learned from the residual of the feature against the principal space, and matched with the original logits by a constant scaling. This virtual logit serves as the indicator of OOD-ness. EGonc has nice theoretical properties that guarantee an overall distinguishable margin between the detection scores for IND and OOD samples. Comprehensive experimental evaluations of EGonc also demonstrate its superiority.

In this work, we study the robustness of two typical terminology translation methods: Placeholder (PH) and Code-Switch (CS), concerning (1) the number of constraints and (2) the target constraint length.We identify that existing terminology constraint test sets, such as IATE, Wiktionary, and TICO, are blind to this issue due to oversimplified constraint settings.To solve it, we create a new challenging test set of English-German, increasing the average constraint count per sentence from 1.1∼1.7 to 6.1 and the length per target constraint from 1.1∼1.2words to 3.4 words.Then we find that PH and CS methods degrade as the number of constraints increases, but they have complementary strengths.Specifically, PH is better at retaining high constraint accuracy but lower translation quality as measured by BLEU and COMET scores.In contrast, CS has the opposite results.Based on these observations, we propose a simple but effective method combining the advantages of PH and CS.This approach involves training a model like PH to predict the term labels, and then during inference replacing those labels with target terminology text like CS, so that the subsequent generation is aware of the target term content.Extensive experimental results show that this approach can achieve high constraint accuracy and translation quality simultaneously, regardless of the number or length of constraints. 1

Node classification is the task of predicting the labels of unlabeled nodes in a graph. State-of-the-art methods based on graph neural networks achieve excellent performance when all labels are available during training. But in real-life, models are of ten applied on data with new classes, which can lead to massive misclassification and thus significantly degrade performance. Hence, developing open-set classification methods is crucial to determine if a given sample belongs to a known class. Existing methods for open-set node classification generally use transductive learning with part or all of the features of real unseen class nodes to help with open-set classification. In this paper, we propose a novel generative open-set node classification method, i.e., G2Pxy, which follows a stricter inductive learning setting where no information about unknown classes is available during training and validation. Two kinds of proxy unknown nodes, inter-class unknown proxies and external unknown proxies are generated via mixup to efficiently anticipate the distribution of novel classes. Using the generated proxies, a closed-set classifier can be transformed into an open-set one, by augmenting it with an extra proxy classifier. Under the constraints of both cross entropy loss and complement entropy loss, G2Pxy achieves superior effectiveness for unknown class detection and known class classification, which is validated by experiments on bench mark graph datasets. Moreover, G2Pxy does not have specific requirement on the GNN architecture and shows good generalizations.

Recently, infrared small target detection has attracted extensive attention. However, due to the small size and the lack of intrinsic features of infrared small targets, the existing methods generally have the problem of inaccurate edge positioning and the target is easily submerged by the background. Therefore, we propose an innovative gradient-guided learning network (GGL-Net). Specifically, we are the first to explore the introduction of gradient magnitude images into the deep learning-based infrared small target detection method, which is conducive to emphasizing the edge details and alleviating the problem of inaccurate edge positioning of small targets. On this basis, we propose a novel dual-branch feature extraction network that utilizes the proposed gradient supplementary module (GSM) to encode raw gradient information into deeper network layers and embeds attention mechanisms reasonably to enhance feature extraction ability. In addition, we construct a two-way guidance fusion module (TGFM), which fully considers the characteristics of feature maps at different levels. It can facilitate the effective fusion of multi-scale feature maps and extract richer semantic information and detailed information through reasonable two-way guidance. Extensive experiments prove that GGL-Net has achieves state-of-the-art results on the public real NUAA-SIRST dataset and the public synthetic NUDT-SIRST dataset.

Modeling videos and image sets by linear subspaces has achieved great success in various visual recognition tasks. However, subspaces constructed from visual data are always notoriously embedded in a high-dimensional ambient space, which limits the applicability of existing techniques. This letter explores the possibility of proposing a geometry-aware framework for constructing lower-dimensional subspaces with maximum discriminative power from high-dimensional subspaces in the supervised scenario. In particular, we make use of Riemannian geometry and optimization techniques on matrix manifolds to learn an orthogonal projection, which shows that the learning process can be formulated as an unconstrained optimization problem on a Grassmann manifold. With this natural geometry, any metric on the Grassmann manifold can theoretically be used in our model. Experimental evaluations on several data sets show that our approach results in significantly higher accuracy than other state-of-the-art algorithms.

This letter proposes a generalized framework with joint normalization that learns lower dimensional subspaces with maximum discriminative power by using Riemannian geometry. We model the similarity/dissimilarity between subspaces using various metrics defined on Grassmannian and formulate dimensionality reduction as a nonlinear constraint optimization problem considering the orthogonalization. To obtain the linear mapping, we derive the components required to perform Riemannian optimization from the original Grassmannian through an orthonormal projection. We respect the Riemannian geometry of the Grassmann manifold and search for this projection directly from one Grassmann manifold to another face-to-face without any additional transformations. In this natural geometry-aware approach, any metric on the Grassmann manifold can theoretically reside in our model. We combine five metrics with our model, and the learning process is treated as an unconstrained optimization problem on a Grassmann manifold. Experiments on several datasets demonstrate that our approach leads to a significant accuracy gain over state-of-the-art methods.

This paper considers the joint geometric and photometric image registration problem. The inverse compositional (IC) algorithm and the efficient second-order minimization (ESM) algorithm are two typical efficient methods applied to the geometric registration problem. Their efficiency stems from the utilization of the group structure of geometric transformations. To allow for photometric variations, the dual IC algorithm (DIC) proposed by Bartoli performs joint geometric and photometric image registration by extending the IC algorithm. The group structures of both geometric and photometric transformations are exploited. Despite the robustness to large photometric variations, DIC is vulnerable to large geometric deformations. The ESM algorithm is extended by Silveira et al. to address photometric variations. In their approach, the photometric transformations are modeled in Euclidean space. Their approach is robust to relatively large geometric and photometric transformations; however, it is not efficient for large photometric variations. We propose a new efficient and robust image registration method by exploiting the non-Euclidean Lie group structure of joint geometric and photometric transformations for both grayscale and color images. The image registration is formulated as a nonlinear least squares problem. In our method, the geometric and photometric transformations are jointly parameterized by their corresponding Lie algebras. Based on this parameterization approach, the second-order approximation strategy of ESM is employed to optimize the joint geometric and photometric parameters. The error function in the nonlinear least squares problem is approximated by a second-order Taylor expansion with respect to joint geometric and photometric parameters without computing the Hessian matrix. For further efficiency, independent convergence criteria for geometric and photometric parameters are used in the iterative optimization process. The superiority of our proposed method over the previous methods, in terms of efficiency, accuracy, and robustness, is demonstrated through extensive experiments on synthetic and real data.

Bilateral filtering has the advantage of keeping the edge well in image smoothing, but the selection of its parameters is difficult. In this paper, we propose an adaptive parameter estimation method for bilateral filtering, by estimating the noise variance automatically through image wavelet decomposition. When adaptive bilateral filtering method is used in Retinex image enhancement, it can either remove image noise or avoid the halo artifacts. The experimental results show that our method is effective.

Previous image clutter metrics were proposed on the thought that clutter was just a perceptual effect, while we identify clutter as both perceptual and cognitive effects. Under this identification, we give a new definition of image clutter metric by analyzing the research results in the fields of visual psychology and psychophysics. According to the definition, we further put forward a DisSIMilarity (DSIM) based image clutter metric, which can also be taken as a kind of HVS-based signal-to-clutter ratio. The earlier image clutter metrics produced limited success in predicting targeting performance mainly since they did not consider brain cognitive characteristics. We develop a brain cognitive dissimilarity measure (BCDM) as a quantitative estimate of the selection weights which are allocated by brain attentional mechanism to affect visual selection processes. A human vision perceptual dissimilarity measure (VPDM), fully embodying vision perceptual properties, is first established between the target and clutter images, and then we utilize the BCDM between the two images as selection weights to pool the VPDM to be a clutter metric, which can be called DSIM metric. The metric is tested in Search_2 dataset provided by TNO Human Factors Research Institute of Netherlands. Error analysis and correlation tests demonstrate that the DSIM metric makes a more significant improvement than previously proposed metrics in predicting 62 observers' targeting performances including detection probability, false alarm probability and search time.

Discrete wavelet transform (DWT) is sensitive to the translation/shift of input signals, so its effectiveness could be negatively impacted when we encounter translation among signals. To deal with such drawbacks, this paper proposes redundant DWT(RDWT) based method to achieve image registration, translation invariant wavelet feature extraction and face recognition. We select a representative face from each person to form a reference face set and perform DWT on it. For each test face, we perform RDWT and compare its redundant horizontal and vertical details with the corresponding details obtained from the reference face. The reference face that is the most similar to the test face is determined to be the recognized face. Experiments on Yaleface database prove the effectiveness of our RDWT based method.