Automated captioning for remote sensing imagery often struggles to balance the high descriptive power of large models with the deployment feasibility of smaller ones. To bridge this gap, this paper introduces ONIR, a LLM-efficient, tag-guided framework that empowers compact language models (1-3B parameters) to achieve state-of-the-art captioning accuracy. Specifically, the proposed approach synthesizes a large-scale pseudo-caption dataset by leveraging GPT-4O on existing segmentation benchmarks. Explicit semantic tags are then extracted to train a multi-label Contrastive Language-Image Pre-Training (CLIP) encoder, providing interpretable visual guidance. To maintain parameter efficiency, the architecture incorporates a simple Multilayer Perceptron (MLP) bridge and a two-stage LoRA fine-tuning strategy. Extensive experiments on standard benchmark dataset, such as UCM and Sydney Captions, demonstrate that ONIR significantly outperforms models up to four times its size (7-13B). By combining superior performance with computational efficiency and tag-based controllability, ONIR offers a highly practical solution for real-world remote sensing applications.

Visual Question Answering (VQA) in remote sensing (RS) is pivotal for interpreting Earth observation data. However, existing RS VQA datasets are constrained by limitations in annotation richness, question diversity, and the assessment of specific reasoning capabilities. This paper introduces Remote Sensing Vision Language Model Question Answering (RSVLM-QA) dataset, a new large-scale, content-rich VQA dataset for the RS domain. RSVLM-QA is constructed by integrating data from several prominent RS segmentation and detection datasets: WHU, LoveDA, INRIA, and iSAID. We employ an innovative dual-track annotation generation pipeline. Firstly, we leverage Large Language Models (LLMs), specifically GPT-4.1, with meticulously designed prompts to automatically generate a suite of detailed annotations including image captions, spatial relations, and semantic tags, alongside complex caption-based VQA pairs. Secondly, to address the challenging task of object counting in RS imagery, we have developed a specialized automated process that extracts object counts directly from the original segmentation data; GPT-4.1 then formulates natural language answers from these counts, which are paired with preset question templates to create counting QA pairs. RSVLM-QA comprises 13,820 images and 162,373 VQA pairs, featuring extensive annotations and diverse question types. We provide a detailed statistical analysis of the dataset and a comparison with existing RS VQA benchmarks, highlighting the superior depth and breadth of RSVLM-QA's annotations. Furthermore, we conduct benchmark experiments on Six mainstream Vision Language Models (VLMs), demonstrating that RSVLM-QA effectively evaluates and challenges the understanding and reasoning abilities of current VLMs in the RS domain. We believe RSVLM-QA will serve as a pivotal resource for the RS VQA and VLM research communities, poised to catalyze advancements in the field. The dataset, generation code, and benchmark models are publicly available at https://github.com/StarZi0213/RSVLM-QA.

Abstract Multi-way data analysis has become an essential tool for capturing underlying structures in higher-order data sets where standard two-way analysis techniques often fail to discover the hidden correlations between variables in multi-way data. We propose a multi-objective variational autoencoder (MO-VAE) method for smart infrastructure damage detection and diagnosis in multi-way sensing data based on the reconstruction probability of autoencoder deep neural network (ADNN). Our method fuses data from multiple sensors in one ADNN at which informative features are being extracted and utilized for damage identification. It generates probabilistic anomaly scores to detect damage, asses its severity and further localize it via a new localization layer introduced in the ADNN. We evaluated our method on multi-way laboratory-based and real-life structural datasets in the area of structural health monitoring for damage diagnosis purposes. The data was collected from our deployed data acquisition system on a cable-stayed bridge in Western Sydney, a reinforced concrete cantilever beam which replicates one of the major structural components on the Sydney Harbour Bridge and a laboratory based building structure obtained from Los Alamos National Laboratory (LANL). Experimental results show that the proposed method can accurately detect structural damage. It was also able to estimate the different levels of damage severity, and capture damage locations in an unsupervised aspect. Compared to the state-of-the-art approaches, our proposed method shows better performance in terms of damage detection and localization.

Class imbalance occurs in many real-world applications, including image classification, where the number of images in each class differs significantly. With imbalanced data, the generative adversarial networks (GANs) leans to majority class samples. The two recent methods, Balancing GAN (BAGAN) and improved BAGAN (BAGAN-GP), are proposed as an augmentation tool to handle this problem and restore the balance to the data. The former pre-trains the autoencoder weights in an unsupervised manner. However, it is unstable when the images from different categories have similar features. The latter is improved based on BAGAN by facilitating supervised autoencoder training, but the pre-training is biased towards the majority classes. In this work, we propose a novel Conditional Variational Autoencoder with Balanced Pre-training for Generative Adversarial Networks (CAPGAN) as an augmentation tool to generate realistic synthetic images. In particular, we utilize a conditional convolutional variational autoencoder with supervised and balanced pre-training for the GAN initialization and training with gradient penalty. Our proposed method presents a superior performance of other state-of-the-art methods on the highly imbalanced version of MNIST, Fashion-MNIST, CIFAR-10, and two medical imaging datasets. Our method can synthesize high-quality minority samples in terms of Fréchet inception distance, structural similarity index measure and perceptual quality. The source code is available at https://github.com/alibraytee/CAPGAN.

Zero-shot learning (ZSL) aims to recognize instances belonging to unseen categories which are not available at training time. Previous ZSL models learn a projection function from the visual feature space to a semantic space which contains a description of the categories. The semantic attributes are often correlated with each other at the semantic space and it is not appropriate to learn them independently. Existing ZSL methods are designed to work on complete descriptions of the semantic attributes. However, because these attributes are human-designed values, they might be incomplete or contains noisy values which may affect the recognition performance of many existing ZSL models. This paper proposes a novel zero-shot learning approach (ZSL-MSA) to handle missing and noisy semantic attributes during the training process. Significantly, the proposed method learns a supplementary attribute matrix by exploiting the attribute correlation. The proposed method also learns the relevant feature coefficients in the projection matrix to identify the correlated attribute space. Th proposed method also adopts l1regularization norm to select the relevant sparse features. A constrained optimization function is formulated and solved using the accelerated proximal gradient method. Extensive experiments on three benchmark datasets using ZSL and generalized ZSL demonstrate the effectiveness of the proposed method.

With the recent surge in threats to public safety, the security focus of several organizations has been moved towards enhanced intelligent screening systems. Conventional X-ray screening, which relies on the human operator is the best use of this technology, allowing for the more accurate identification of potential threats. This paper explores X-ray security imagery by introducing a novel approach that generates realistic synthesized data, which opens up the possibility of using different settings to simulate occlusion, radiopacity, varying textures, and distractors to generate cluttered scenes. The generated synthetic data is effective in the training of deep networks. It allows better generalization on training data to deal with domain adaptation in the real world. The extensive set of experiments in this paper provides evidence for the efficacy of synthetic datasets over human-annotated datasets for automated X-ray security screening. The proposed approach outperforms the state-of-the-art approach for a diverse threat object dataset on mean Average Precision (mAP) of region-based detectors and classification/regression-based detectors.

Anomaly detection in X-ray security screening systems has earned a lot of interests in recent years and has attracted many researchers working in the area of machine learning. With the advances in computing technology, it is becoming more feasible to develop an approach for automated anomaly detection in security screening systems based on images collected via Xray machines. Analyzing these X-ray images and constructing a detection model is considered as a challenging problem because of the lack or limited number of samples of anomalous objects. This paper presents a novel tensor based learning method for anomaly detection in X-ray security screening systems based on tensor analysis augmented with one-class classification model. Our method initially performs data fusion of multi-angle scanned images in a tensor data structure from where we extract the informative features. Further, it constructs a one-class support vector machine model using these features to detect anomalies. We evaluate this approach using two image-based datasets and one real X-ray security baggage data collected from Sydney airport. The results show that our tensor based learning method outperforms other state-of-the-art approaches.

Multi-label classification is defined as the problem of identifying the multiple labels or categories of new observations based on labeled training data. Multi-labeled data has several challenges, including class imbalance, label correlation, incomplete multi-label matrices, and noisy and irrelevant features. In this article, we propose an integrated multi-label classification approach with incomplete label space and class imbalance (ML-CIB) for simultaneously training the multi-label classification model and addressing the aforementioned challenges. The model learns a new label matrix and captures new label correlations, because it is difficult to find a complete label vector for each instance in real-world data. We also propose a label regularization to handle the imbalanced multi-labeled issue in the new label, and l 1 regularization norm is incorporated in the objective function to select the relevant sparse features. A multi-label feature selection (ML-CIB-FS) method is presented as a variant of the proposed ML-CIB to show the efficacy of the proposed method in selecting the relevant features. ML-CIB is formulated as a constrained objective function. We use the accelerated proximal gradient method to solve the proposed optimisation problem. Last, extensive experiments are conducted on 19 regular-scale and large-scale imbalanced multi-labeled datasets. The promising results show that our method significantly outperforms the state-of-the-art.

The one-class support vector machines with Gaussian kernel function is a promising machine learning method which have been employed extensively in the area of anomaly detection. However, generalization performance of OCSVM is profoundly influenced by its Gaussian model parameter σ. This paper proposes a new algorithm named Edged Support Vector (ESV) for tuning the Gaussian model parameter. The semantic idea of this algorithm is based on inspecting the spatial locations of the selected support vector samples. The algorithm selects the optimal value of σ which leads to a decision boundary that has all its support vectors reside on the surface of the training data (i.e. edged support vector). A support vector is identified as an edge sample by constructing a hyperplane with its k-nearest neighbour samples using a hard margin linear support vector machine. The algorithm was successfully validated using two real world sensing datasets, one collected from a lab specimen which was replicated a jack arch from the Sydney Harbour Bridge, and another one collected from sensors mounted on vehicles for road condition assessment. Results show that the designed ESV algorithm is an appropriate choice to identify the optimal value of σ for OCSVM.

High-dimensional multi-labeled data contain instances, where each instance is associated with a set of class labels and has a large number of noisy and irrelevant features. Feature selection has been shown to have great benefits in improving the classification performance in machine learning. In multi-label learning, to select the discriminative features among multiple labels, several challenges should be considered: interdependent labels, different instances may share different label correlations, correlated features, and missing and flawed labels. This work is part of a project at The Children's Hospital at Westmead (TB-CHW), Australia to explore the genomics of childhood leukaemia. In this paper, we propose a CMFS (Correlated- and Multi-label Feature Selection method), based on non-negative matrix factorization (NMF) for simultaneously performing feature selection and addressing the aforementioned challenges. Significantly, a major advantage of our research is to exploit the correlation information contained in features, labels and instances to select the relevant features among multiple labels. Furthermore, l2,1 -norm regularization is incorporated in the objective function to undertake feature selection by imposing sparsity on the feature matrix rows. We employ CMFS to decompose the data and multi-label matrices into a low-dimensional space. To solve the objective function, an efficient iterative optimization algorithm is proposed with guaranteed convergence. Finally, extensive experiments are conducted on high-dimensional multi-labeled datasets. The experimental results demonstrate that our method significantly outperforms state-of-the-art multi-label feature selection methods.

Traditional feature selection techniques are used to identify a subset of the most useful features, and consider the rest as unimportant, redundant or noisy. In the presence of highly correlated features, many variable selection methods consider correlated features as redundant and need to be removed. In this paper, a novel supervised feature selection algorithm SCANMF is proposed by jointly integrating correlation analysis and structural analysis of the balanced supervised non-negative matrix factorization (NMF). Furthermore, ℓ2,1-norm minimization constraint is incorporated into the objective function to guarantee sparsity in the feature matrix rows and reduce noisy features. Our algorithm exploits the discriminative information, feature combinations, and the original features in the context of a supervised NMF method which can be beneficial for both classification and interpretation. An efficient iterative algorithm is designed to solve the constrained optimization problem with guaranteed convergence. Finally, a series of extensive experiments are conducted on 8 complex datasets. Promising results using multiple classifiers demonstrate the effectiveness and efficiency of our algorithm over state-of-the-art methods.

Supervised feature extraction methods have received considerable attention in the data mining community due to their capability to improve the classification performance of the unsupervised dimensionality reduction methods. With increasing dimensionality, several methods based on supervised feature extraction are proposed to achieve a feature ranking especially on microarray gene expression data. This paper proposes a method with twofold objectives: it implements a balanced supervised non-negative matrix factorization (BSNMF) to handle the class imbalance problem in supervised non-negative matrix factorization techniques. Furthermore, it proposes an accurate gene ranking method based on our proposed BSNMF for microarray gene expression datasets. To the best of our knowledge, this is the first work to handle the class imbalance problem in supervised feature extraction methods. This work is part of a Human Genome project at The Children's Hospital at Westmead (TB-CHW), Australia. Our experiments indicate that the factorized components using supervised feature extraction approach have more classification capability than the unsupervised one, but it drastically fails at the presence of class imbalance problem. Our proposed method outperforms the state-of-the-art methods and shows promise in overcoming this concern.

Class imbalanced data is a common problem for predictive modelling in domains such as bioinformatics. It occurs when the distribution of classes is not uniform among samples and results in a biased prediction of learning towards majority classes. In this study, we propose the ABC-Sampling algorithm based on a swarm optimization method called Artificial Bee Colony, which models the natural foraging behaviour of honeybees. Our algorithm lessens the effects of imbalanced classes by selecting the most informative majority samples using a forward search and storing them in a ranked subset. Then we construct a balanced dataset with a planned undersampling strategy to extract the most frequent majority samples from the top ranked subset and combine them with all minority samples. Our algorithm is superior to a state-of-the-art method on nine benchmark datasets with various levels of imbalance ratios.