ConvNets and Vision Transformers (ViTs) have been widely used for change detection, though they exhibit limitations: long-range dependencies are not effectively captured by the former, while the latter are associated with high computational demands. Vision Mamba, based on State Space Models, has been proposed as an alternative, yet has been primarily utilized as a feature extraction backbone. In this work, the Change State Space Model (CSSM) is introduced as a task-specific approach for change detection, designed to focus exclusively on relevant changes between bi-temporal images while filtering out irrelevant information. Through this design, the number of parameters is reduced, computational efficiency is improved, and robustness is enhanced. CSSM is evaluated on three benchmark datasets, where superior performance is achieved compared to ConvNets, ViTs, and Mamba-based models, at a significantly lower computational cost. The code will be made publicly available at https://github.com/Elman295/CSSM upon acceptance.

Building instance segmentation is a critical task for urban planning. It has been extensively studied through state-of-the-art instance segmentation models, and great advances have been reported. However, the issue of domain shift referring to disparities between training and target data distributions remains elusive. Although domain adaptation can help tackle it, it still requires access to target domain data. In this paper, we explore the problem of building extraction in the domain generalization setting, where no access to either target labels or target data is assumed. Mask R-CNN is equipped with image, instance, and pixel-level domain adversarial modules in order to encourage the extraction of domain-invariant features. The preliminary results obtained with cross-continent domain generalization are promising. For the sake of reproducibility, the code and models are publicly available on this website: https://github.com/efkandurakli/DGMaskRCNN.

Accurately segmenting region boundaries in complex and high resolution remote sensing scenes, with often a large number of relatively small structures and objects remains a challenge; since conventional loss functions such as Cross-Entropy and Intersection-over-Union often neglect boundary precision and focus instead on the alignment of the entire estimated region. This paper presents a new distance transform-based loss function designed especially to focus on boundary quality enhancement. It is validated with the ISPRS very high spatial resolution Vaihingen and Potsdam remote sensing datasets using a U-Net model with a ResNet-50 encoder. Preliminary results show that the proposed loss function outperforms widely used loss functions across multiple evaluation metrics.

Floods are one of the most common natural disasters, causing fatalities and severe economic and environmental impacts, directly affecting agriculture, urban infrastructure, and transportation networks. Hence, it is of utmost importance that flooded areas are efficiently and effectively identified in the aftermath. Synthetic aperture radar (SAR) images are invaluable to this end, since the amount of microwave energy reflected from water is less than that from land, due to its low surface roughness and lack of apparent texture. In this study, we explore the combination of histograms with deep neural networks for the purpose of flood mapping. The proposed histogram extraction layers, specifically designed for SAR content, are integrated into deep segmentation neural networks and are tested on two real SAR datasets. Experimental results have shown that histogram layers integrated into deep segmentation neural networks improve the performance up to 6% in terms of intersection over union (IoU) with a negligible increase in the number of learnable parameters. The code of the work will be available athttps://github.com/ilterturkmenli/HistSegNet.

Domain generalisation (i.e. out-of-distribution generalisation) is an open problem in machine learning, where the goal is to train a model via one or more source domains, that will generalise well to unknown target domains. While the topic is attracting increasing interest, it has not been studied in detail in the context of object detection. The established approaches all operate under the covariate shift assumption, where the conditional distributions are assumed to be approximately equal across source domains. This is the first paper to address domain generalisation in the context of object detection, with a rigorous mathematical analysis of domain shift, without the covariate shift assumption. We focus on improving the generalisation ability of object detection by proposing new regularisation terms to address the domain shift that arises due to both classification and bounding box regression. Also, we include an additional consistency regularisation term to align the local and global level predictions. The proposed approach is implemented as a Domain Generalised Faster R-CNN and evaluated using four object detection datasets which provide domain metadata (GWHD, Cityscapes, BDD100K, Sim10K) where it exhibits a consistent performance improvement over the baselines. All the codes for replicating the results in this paper can be found at https://github.com/karthikiitm87/domain-generalisation.git

Real-time vision in robotics plays an important role in localising and recognising objects. Recently, deep learning approaches have been widely used in robotic vision. However, most of these approaches have assumed that training and test sets come from similar data distributions, which is not valid in many real world applications. This study proposes an approach to address domain generalisation (i.e. out-of-distribution generalisation, OODG) where the goal is to train a model via one or more source domains, that will generalise well to unknown target domains using single stage detectors. All existing approaches which deal with OODG either use slow two stage detectors or operate under the covariate shift assumption which may not be useful for real-time robotics. This is the first paper to address domain generalisation in the context of single stage anchor free object detector FCOS without the covariate shift assumption. We focus on improving the generalisation ability of object detection by proposing new regularisation terms to address the domain shift that arises due to both classification and bounding box regression. Also, we include an additional consistency regularisation term to align the local and global level predictions. The proposed approach is implemented as a Domain Generalised Fully Convolutional One Stage (DGFCOS) detection and evaluated using four object detection datasets which provide domain metadata (GWHD, Cityscapes, BDD100K, Sim10K) where it exhibits a consistent performance improvement over the baselines and is able to run in real-time for robotics.

This study addresses the explainability challenges of deep-learning models in the context of hyperspectral remote sensing image classification. Three prominent explainable artificial intelligence methods, namely GradCAM, GradCAM++, and Guided Backpropagation, have been employed in order to comprehend the decision-making process of a typical convolutional neural network model during spatial-spectral hyperspectral image classification. The experiments that have been conducted investigate the impact of pixel patch sizes on spatial attention, as well as spectral band importance. The findings provide insights into the behavior of both convolutional neural networks, as well as the comparative performance of explainability techniques.

Domain adaptation is one of the prominent strategies for handling both the scarcity of pixel-level ground truth and the domain shift, that is widely encountered in large-scale land use/cover map calculation. Studies focusing on adversarial domain adaptation via re-styling source domain samples, commonly through generative adversarial networks (GANs), have reported varying levels of success, yet they suffer from semantic inconsistencies, visual corruptions, and often require a large number of target domain samples. In this letter, we propose a new lightweight unsupervised domain adaptation (UDA) method for the semantic segmentation of very high-resolution remote sensing images, based on an image-to-image translation (I2IT) approach, via an encoder–decoder strategy where latent content representations are mixed across domains, and a perceptual network module and loss function enforce visual semantic consistency. We show through cross-domain comparative experiments that it: 1) leads to semantically consistent images; 2) can operate with a single target domain sample (i.e., one-shot); and 3) at a fraction of the number of parameters required from the state-of-the-art methods, while still outperforming them. Code is available at github.com/Sarmadfismael/RSOS_I2I.

The combination of Convolutional Neural Networks (CNN) with Long Short Term Memory (LSTM) networks in the form of CNN-LSTMs, is one of the currently widely used temporal data series processing approaches. It harnesses the CNN's feature extraction ability along with the LSTM's capacity to account for sequential dependencies. Mathematical morphology on the other hand is known for its spatial analysis potential. In this study, we explore the combination of morphological neural networks (MNNs) with LSTMs, in the form of MNN-LSTMs, and apply it to the problem of crop classification from multi-spectral/temporal remote sensing images. The explored method is tested with two real datasets, where it exhibits either superior or comparable performance to CNN-LSTMs and other state-of-the-art alternative approaches.

Estimating reliably chlorophyll-a (Chl-a) concentration from remote sensing images constitutes a vastly superior alternative to field measurements. To this end, spectral pixel signatures are used commonly for developing regression models. Spatial information has been traditionally ignored in this context, as Chl-a concentration is a spatially localized measurement, and sensors’ spatial resolutions have been relatively low in the past. However, the increased spatial resolution of newer satellites and a recent study have given strong indications that spatial–spectral description can boost estimation performance. Consequently, in this letter, we address the problem of Chl-a estimation from remote sensing images using attribute profiles, one of the paramount spatial–spectral description tools. We further propose an original technique to remove their cumbersome threshold requirement via operating on each pixel’s “attribute lineage.” We validate our approach with multispectral Sentinel-2 images, and a data set formed by field measurements spanning almost two years over Lake Balik (Turkey). We show that the proposed method outperforms various alternatives in terms of regression performance, across multiple experimental setups, and finally, we highlight a validation malpractice encountered often in the field of water quality estimation.

In this letter, we explore harnessing the power of regression-oriented convolutional neural networks (CNN) for the assessment of surface water quality from remote sensing images. They are used to estimate the chlorophyll-a concentration of Lake Balik (Turkey), through multispectral Sentinel-2 images. The proposed approach is tested with a data set$(n=320)$ofin situ Chl-ameasurements acquired during 2017–2019. We investigate both 2-D and 3-D convolution strategies and report the results of a series of rigorous validation experiments, aiming to measure both spatial, short-term, and long-term temporal generalization performance, thus highlighting validation misconduct encountered often in the state-of-the-art. The regression-oriented CNNs outperform various alternatives, in all generalization scenarios with performances reaching 0.95, 0.93, and 0.76 in terms of$R^{2}$, respectively. It has been deployed as an online service producing regularly water quality maps for the lake under study as the first of its kind in Turkey.

It is well known in the machine learning community that the ensembles of neural networks outperform the respective individual networks; hence recent work on aerial scene classification has been trending toward various network fusion strategies. However, training multiple deep networks can be computationally expensive. “Snapshot ensembling” has recently appeared as an alternative and claims to provide the performance of network ensembles at the cost of a single network's training. In this letter, we present the results of a comparative study on deep network ensembles in the context of aerial scene classification, where homogeneous, heterogeneous, and snapshot ensembling strategies are explored with both DenseNet and Inception networks; contrary to the existing work, we employ model fusion at the last convolutional layer's level. The explored approaches are validated with the two largest and most challenging data sets available (AID and NWPU-RESISC45), and state-of-the-art results are achieved on both.

Attribute profiles (APs) are among the most prominent spectral-spatial pixel description tools, widely employed with optical, and hyperspectral images in particular. Their use, however, with the SAR data has been very limited. In this letter, we investigate the use of the recently developed feature attribute profiles (FPs) for the pixel-based classification of SAR images. Compared to low-level features such as intensity and amplitude, FPs are shown to provide high-level features incorporating successfully the spatial information of the pixels. Based on the classification results obtained on real TerraSAR-X images, it is shown that the FPs are capable of more accurately classifying the pixels compared to conventional and widely used SAR feature extraction methods such as gray-level co-occurrence matrices, histograms of oriented gradients, local binary patterns, and Gabor filters.

Morphological attribute profiles are among the most prominent spatial-spectral pixel description methods. They are efficient, highly flexible multiscale tools that operate at the connected component level of images. One of their few yet significant drawbacks is their need for a predefined threshold set. As such there have been multiple attempts for computing thresholds with minimal or no supervision with various levels of success. In this paper, a radically different approach is taken and a new way is presented, circumventing the need for thresholds while harnessing the descriptive power of the hierarchical tree representation underlying the attribute profiles. The introduced approach is validated with two datasets and two attributes, where it exhibits either comparable or superior performance to manual and automatic threshold based attribute profiles.

The motivation of this paper is to conduct a comparative study on remote sensing image classification using the morphological attribute profiles (APs) and feature profiles (FPs) generated from different types of tree structures. Over the past few years, APs have been among the most effective methods to model the image's spatial and contextual information. Recently, a novel extension of APs called FPs has been proposed by replacing pixel gray-levels with some statistical and geometrical features when forming the output profiles. FPs have been proved to be more efficient than the standard APs when generated from component trees (max-tree and min-tree). In this work, we investigate their performance on the inclusion tree (tree of shapes) and partition trees (alpha tree and omega tree). Experimental results from both panchromatic and hyperspectral images again confirm the efficiency of FPs compared to APs.

The amount of training data that is required to train a classifier scales with the dimensionality of the feature data. In hyperspectral remote sensing (HSRS), feature data can potentially become very high dimensional. However, the amount of training data is oftentimes limited. Thus, one of the core challenges in HSRS is how to perform multiclass classification using only relatively few training data points. In this letter, we address this issue by enriching the feature matrix with synthetically generated sample points. These synthetic data are sampled from a Gaussian mixture model (GMM) fitted to each class of the limited training data. Although the true distribution of features may not be perfectly modeled by the fitted GMM, we demonstrate that a moderate augmentation by these synthetic samples can effectively replace a part of the missing training samples. Doing so, the median gain in classification performance is 5% on two datasets. This performance gain is stable for variations in the number of added samples, which makes it easy to apply this method to real-world applications.

This paper introduces an extension of morphological attribute profiles (APs) by extracting their local features. The so-called local feature-based APs (LFAPs) are expected to provide a better characterization of each APs' filtered pixel (i.e., APs' sample) within its neighborhood, and hence better deal with local texture information from the image content. In this paper, LFAPs are constructed by extracting some simple first-order statistical features of the local patch around each APs' sample such as mean, standard deviation, and range. Then, the final feature vector characterizing each image pixel is formed by combining all local features extracted from APs of that pixel. In addition, since the self-dual APs (SDAPs) have been proved to outperform the APs in recent years, a similar process will be applied to form the local feature-based SDAPs (LFSDAPs). In order to evaluate the effectiveness of LFAPs and LFSDAPs, supervised classification using both the random forest and the support vector machine classifiers is performed on the very high resolution Reykjavik image as well as the hyperspectral Pavia University data. Experimental results show that LFAPs (respectively, LFSDAPs) can considerably improve the classification accuracy of the standard APs (respectively, SDAPs) and the recently proposed histogram-based APs.

The present paper introduces an extension of attribute profiles (APs) by extracting their local features. The so-called local feature-based attribute profiles (LFAPs) are expected to provide a better characterization of each APs' filtered pixel (i.e. APs' sample) within its neighborhood, hence better deal with local texture information from the image's content. In this work, LFAP is constructed by extracting some simple first-order statistical features of the local patch around each APs' sample such as mean, standard deviation, range, etc. Then, the final feature vector characterizing each image pixel is formed by combining all local features extracted from APs of that pixel. In order to evaluate the effectiveness of the proposed technique, supervised classification using Random Forest classifier is performed on the VHR panchromatic Reykjavik image. Experimental results show that LFAPs can considerably improve the classification accuracy of the standard APs and the recently proposed histogram-based APs.

Effective spatial-spectral pixel description is of crucial significance for the classification of hyperspectral remote sensing images. Attribute profiles are considered as one of the most prominent approaches in this regard, since they can capture efficiently arbitrary geometric and spectral properties. Lately though, the advent of deep learning in its various forms has also led to remarkable classification performances by operating directly on hyperspectral input. In this letter, we explore the collaboration potential of these two powerful feature extraction approaches. Specifically, we propose a new strategy for hyperspectral image classification, where attribute filtered images are stacked and provided as input to convolutional neural networks. Our experiments with two real hyperspectral remote sensing data sets show that the proposed strategy leads to a performance improvement, as opposed to using each of the involved approaches individually.

Morphological attribute profiles are among the most prominent spectral-spatial pixel description methods. They are efficient, effective, and highly customizable multiscale tools based on hierarchical representations of a scalar input image. Their application to multivariate images in general and hyperspectral images in particular has been so far conducted using the marginal strategy, i.e., by processing each image band (eventually obtained through a dimension reduction technique) independently. In this paper, we investigate the alternative vector strategy, which consists in processing the available image bands simultaneously. The vector strategy is based on a vector-ordering relation that leads to the computation of a single max and min tree per hyperspectral data set, from which attribute profiles can then be computed as usual. We explore known vector-ordering relations for constructing such max trees and, subsequently, vector attribute profiles and introduce a combination of marginal and vector strategies. We provide an experimental comparison of these approaches in the context of hyperspectral classification with common data sets, where the proposed approach outperforms the widely used marginal strategy.

Morphological profiles have been established during the past decade as one of the principal spatial-spectral pixel description methods. Attribute profiles (APs) in particular have recently emerged as their more efficient generalization, enabling the description of image components through arbitrary parametric features, thus leading to more flexible, complete, and accurate content representations. More precisely, their adaptation to hyperspectral images has been realized through their independent application to an image's bands, after some form of spectral dimension reduction, hence resulting in extended APs. In this letter, a variation of this strategy is explored, consisting of using all of the available image bands simultaneously, during the attribute computation of a connected image component. Thus, the use of a wider array of attributes is enabled, targeting collections of vector pixel values instead of scalars. Specifically, a couple of new multidimensional attributes are investigated, namely, the higher-dimensional spread and higher-dimensional dispersion, describing, respectively, the extent and homogeneity of a multidimensional pixel value distribution. Their practical interest is validated through two common hyperspectral data sets, where they systematically achieve superior classification performance.

Despite the popularity of mathematical morphology with remote sensing image analysis, its application to hyperspectral data remains problematic. The issue stems from the need to impose a complete lattice structure on the multi-dimensional pixel value space, that requires a vector ordering. In this article, we introduce such a supervised ordering relation, which conversely to its alternatives, has been designed to be image-specific and exploits the spectral purity of pixels. The practical interest of the resulting multivariate morphological operators is validated through classification experiments where it achieves state-of-the-art performance.

In this paper, we present the results of applying global morphological texture descriptors to the problem of content-based remote sensing image retrieval. Specifically, we explore the potential of recently developed multiscale texture descriptors, namely, the circular covariance histogram and the rotation-invariant point triplets. Moreover, we introduce a couple of new descriptors, exploiting the Fourier power spectrum of the quasi-flat-zone-based scale space of their input. The descriptors are evaluated with the UC Merced Land Use-Land Cover data set, which has been only recently made public. The proposed approach is shown to outperform the best known retrieval scores, despite its shorter feature vector length, thus asserting the practical interest of global content descriptors as well as of mathematical morphology in this context.

Although plant recognition has become an increasingly popular research topic, it remains nonetheless a scientific and technical challenge. Besides all the difficulties of classic object recognition, such as illumination, viewpoint and scale variations, plants can additionally exhibit visual changes depending on their age and condition, thus demanding a specialized approach. In this paper, we present two descriptors based on mathematical morphology; the first consists of the computation of morphological covariance on the leaf contour profile and the second is an extension of the recently introduced circular covariance histogram, capturing leaf venation characteristics. The effectiveness of both descriptors has been validated with the ImageClef'12 plant identification dataset.

Placed within the context of content-based image retrieval, we study in this paper the potential of morphological operators as far as color description is concerned, a booming field to which the morphological framework, however, has only recently started to be applied. More precisely, we present three morphology-based approaches, one making use of granulometries independently computed for each subquantized color and two employing the principle of multiresolution histograms for describing color, using respectively morphological levelings and watersheds. These new morphological color descriptors are subsequently compared against known alternatives in a series of experiments, the results of which assert the practical interest of the proposed methods.