Accurate perception of the vehicle's 3D surroundings, including fine-scale road geometry, such as bumps, slopes, and surface irregularities, is essential for safe and comfortable vehicle control. However, conventional monocular depth estimation often oversmooths these features, losing critical information for motion planning and stability. To address this, we introduce Gamma-from-Mono (GfM), a lightweight monocular geometry estimation method that resolves the projective ambiguity in single-camera reconstruction by decoupling global and local structure. GfM predicts a dominant road surface plane together with residual variations expressed by$\gamma$, a dimensionless measure of vertical deviation from the plane, defined as the ratio of a point's height above it to its depth from the camera, and grounded in established planar parallax geometry. With only the camera's height above ground, this representation deterministically recovers metric depth via a closed form, avoiding full extrinsic calibration and naturally prioritizing nearroad detail. Its physically interpretable formulation makes it well suited for self-supervised learning, eliminating the need for large annotated datasets. Evaluated on KITTI and the Road Surface Reconstruction Dataset (RSRD), GfM achieves state-of-the-art near-field accuracy in both depth and$\gamma$estimation while maintaining competitive global depth performance. Our lightweight 8.88M-parameter model adapts robustly across diverse camera setups and, to our knowledge, is the first selfsupervised monocular approach evaluated on RSRD.

Scene Graph Generation (SGG) suffers from a long-tailed distribution, where a few predicate classes dominate while many others are underrepresented, leading to biased models that underperform on rare relations. Unbiased-SGG methods address this by implementing debiasing strategies, but often at the cost of spatial understanding—resulting in over-reliance on semantic priors. We introduce Salience-SGG, a novel framework featuring an Iterative Salience Decoder (ISD) that emphasizes triplets with salient spatial structures. To support this, we propose semantic-agnostic salience labels guiding ISD. Evaluations on Visual Genome, Open Images V6, and GQA-200 show that Salience-SGG achieves state-of-the-art performance and improves existing Unbiased-SGG methods in their spatial understanding as demonstrated by the Pairwise Localization Average Precision. Code is available at: https://github.com/runfeng-q/Salience-SGG.

Abstract Machine learning and computer vision methods have a major impact on the study of natural animal behavior, as they enable the (semi-) automatic analysis of vast amounts of video data. Such large-scale analysis is essential not only to mere behavioral research but its applicability spans across a large range of disciplines. Mice are the standard mammalian model system in most behavioral research fields, but the datasets available today to refine such methods focus either on isolated or social behaviors. In contrast, datasets in which animals interact with a physical apparatus, which is highly relevant across disciplines that study learning, are as of yet unavailable. In this work, we present a video dataset of individual mice solving (multi-step) mechanical puzzles, so-called lockboxes. The more than 110 hours of total playtime show their behavior recorded from three different perspectives. As a benchmark for frame-level action classification methods, we provide human-annotated labels for all videos of two different mice, equaling 13% of our dataset. Our action labels provide two levels of complexity, requiring the classification of both what the mouse is doing as well as which object is targeted. Additionally, as an initial comparison against the human-annotated labels, we used two different keypoint (pose) tracking-based action classification frameworks as well as an autoencoder-based framework, which illustrates the challenges of automated labeling of fine-grained behaviors, such as the manipulation of objects. We hope that our work will help accelerate the advancement of automated action and behavior classification in a task-driven environment. Our dataset, including videos and human-annotated labels, is publicly available at https://doi.org/10.14279/depositonce-23850 .

Self-supervised monocular depth estimation (MDE) has gained popularity for obtaining depth predictions directly from videos. However, these methods often produce scaleinvariant results, unless additional training signals are provided. Addressing this challenge, we introduce a novel selfsupervised metric-scaled MDE model that requires only monocular video data and the camera's mounting position, both of which are readily available in modern vehicles. Our approach leverages planar-parallax geometry to reconstruct scene structure. The full pipeline consists of three main networks, a multi-frame network, a singleframe network, and a pose network. The multi-frame network processes sequential frames to estimate the structure of the static scene using planar-parallax geometry and the camera mounting position. Based on this reconstruction, it acts as a teacher, distilling knowledge such as scale information, masked drivable area, metric-scale depth for the static scene, and dynamic object mask to the singleframe network. It also aids the pose network in predicting a metric-scaled relative pose between two subsequent images. Our method achieved state-of-the-art results for the driving benchmark KITTI for metric-scaled depth prediction. Notably, it is one of the first methods to produce self-supervised metric-scaled depth prediction for the challenging Cityscapes dataset, demonstrating its effectiveness and versatility. Project page: https://mono-pp.github.io/

Most deep-learning models for vision tasks rely on RGB images as their primary input layer, assuming the model inherently discovers an optimal representation. In this work, we challenge this assumption and show that image gradients offer a straightforward yet robust representation for multi-frame image restoration. We demonstrate that clusters naturally emerge within gradient patches, indicating improved estimation of the underlying signal. We develop a Video Swin-Transformer model operating in the gradient domain, facilitated by the implementation of two differentiable gradient modules. One module computes image gradients using convolutions with gradient filters, while the other reconstructs an RGB image from its gradient representation using deconvolution in the frequency domain. Additionally, we employ a composite training loss that measures the error both in the color domain and its gradient counterpart. Applied to a multi-frame image restoration task involving the removal of lighting, shadows, and occlusions, our model consistently outperforms RGB-based counterparts without introducing additional parameters, thanks to its gradient regularization. We further apply our framework to various restoration tasks, discussing its advantages and limitations. Qualitative results highlight the model's improved generalization to real-world video scenarios, demonstrating successful adaptation from synthetic image training to real video data deployment.

A method for co-registering pre-surgically acquired tomographic data with a patient at time of surgical intervention is introduced. The system using this method consists of a registration element fixed to the skin of the patient at scan and intervention time, a camera, and a laser pointing device movable on a bow rail circumferencing the body of the patient. Camera and laser bow are firmly mounted and calibrated to each other. Pre-operative planning is done using the tomographic data. After moving the patient out of the gantry, the laser ray is steered to indicate position and orientation of a linear instrument (e.g. a needle) on the patient's skin. It is shown, that the achievable accuracy of the method presented in this paper is sufficient for periradicular therapy.

The complexity of natural scenes makes it challenging to experimentally study the mechanisms behind human gaze behavior when viewing dynamic environments. Historically, eye movements were believed to be driven primarily by space-based attention towards locations with salient features. Increasing evidence suggests, however, that visual attention does not select locations with high saliency but operates on attentional units given by the objects in the scene. We present a new computational framework to investigate the importance of objects for attentional guidance. This framework is designed to simulate realistic scanpaths for dynamic real-world scenes, including saccade timing and smooth pursuit behavior. Individual model components are based on psychophysically uncovered mechanisms of visual attention and saccadic decision-making. All mechanisms are implemented in a modular fashion with a small number of well-interpretable parameters. To systematically analyze the importance of objects in guiding gaze behavior, we implemented five different models within this framework: two purely spatial models, where one is based on low-level saliency and one on high-level saliency, two object-based models, with one incorporating low-level saliency for each object and the other one not using any saliency information, and a mixed model with object-based attention and selection but space-based inhibition of return. We optimized each model's parameters to reproduce the saccade amplitude and fixation duration distributions of human scanpaths using evolutionary algorithms. We compared model performance with respect to spatial and temporal fixation behavior, including the proportion of fixations exploring the background, as well as detecting, inspecting, and returning to objects. A model with object-based attention and inhibition, which uses saliency information to prioritize between objects for saccadic selection, leads to scanpath statistics with the highest similarity to the human data. This demonstrates that scanpath models benefit from object-based attention and selection, suggesting that object-level attentional units play an important role in guiding attentional processing.

Visual prostheses have been introduced to partially restore vision to the blind via visual pathway stimulation. Despite their success, some challenges have been reported by the implanted patients. One of those challenges is the difficulty of object recognition due to the low resolution of the images perceived through these devices. In this paper, a deep learning-based approach combined with image pre-processing is proposed to allow visual prostheses' users to recognize objects in a given scene. The approach simplifies the objects in the scene by displaying the objects in clip art form to enhance object recognition. These clip art images are generated by, first, identifying the objects in the scene using the You Only Look Once (YOLO) deep neural network. The clip art corresponding to each identified object is then retrieved via Google Images. Three experiments were conducted to measure the success of the proposed approach using simulated prosthetic vision. Our results reveal a remarkable decrease in the recognition time, increase in the recognition accuracy and confidence level when using the clip art representation as opposed to using the actual images of the objects. These results demonstrate the utility of object simplification in enhancing the perception of images in prosthetic vision.

Despite the growing interest among researchers, satellite-based prediction of soil salinity remains highly uncertain. The improvements in prediction accuracy reported in previous studies are usually limited to a single area. We performed a meta-analysis of regional satellite-based soil salinity predictions combined within situsoil sampling and machine learning. Based on$R^{2}$and root-mean-square error (RMSE) collected, we evaluated the effects of various features on the model accuracy and established a Bayesian network to evaluate the joint causal effect of multifeatures. Most significant differences were found in soil sampling schemes and characteristics of the study area, including the mean and variability (averaged$R^{2}$of 0.75 for soil sample sets with lower salinity variation and 0.62 for others) of the salinity, climate type ($R^{2}$of 0.64 in arid areas and 0.74 in others), soil texture ($R^{2}$of 0.66 in sandy areas and 0.57 in others), and the interval between sampling date and satellite data acquisition date ($R^{2}$of 0.53 under the condition of over 15 days and 0.65 in others). Generally, using different satellite data has limited effects on model performance among which Sentinel-2 performed better ($R^{2} $= 0.72) than Landsat ($R^{2} $= 0.66). The sampling of subsamples for each sample should focus on their subpixel-scale spatial heterogeneity across satellite data rather than the number of subsamples. It is also necessary to select appropriate vegetation and salinity indices for different satellite data under different vegetation conditions. Among algorithms, random forests ($R^{2} $= 0.70) and support vector machines ($R^{2} $= 0.71) performed best.

In the past few decades, the computer vision domain has achieved outstanding success in learning 3D shapes for classification, segmentation and image-based reconstruction. However, deep networks are less explored for the generative task of obtaining new 3D shapes from the learned representation. This problem becomes more prominent for 3D shapes represented as surface meshes, mainly because the mesh structure lacks regularity, an essential property for training deep generative networks. In this work, we remedy this problem by proposing a generative icosahedral mesh convolutional network (GlenIcoNet) that learns data distribution of surface meshes. Our end-to-end trainable network learns semantic representations using 2D convolutional filters on the regularized icosahedral meshes. During inference, GenIcoNet can be used to generate new geometrically valid shapes directly as surface meshes. Our experiments for interpolation of latent space demonstrate that GenIcoNet is able to outperform networks trained on intermediate surface mesh representations. The variational autoencoder architecture of GenIcoNet learns meaningful representation which is numerically stable w.r.t. small perturbations, allows performing exploration and combination of surface meshes to generate new meaningful shapes, while maintaining the essential property of mesh manifoldness. Our code is available at https://github.com/hrdkjain/GenIcoNet

In this paper, we propose a method that allows to simultaneously extract urban structures and estimate scatterer heights and reflectivities from Tomographic SAR (TomoSAR) data. This method relies on a deep convolutional neural network and does not require any manual labelling. Training data is produced by simulating many simple ground/building scenes according to the acquisition geometry of the real dataset to be processed. It does not require prior tomographic focussing and operates on single-look complex (SLC) stacks. Our experiments lead to good agreement between the estimated and true values on simulated data and show promising results on real F-SAR data (DLR).

With the increasing importance of monitoring urban areas, the question arises which sensors are best suited to solve the corresponding challenges. This letter proposes novel node tests within the random forest (RF) framework, which allows them to apply them to optical RGB images, hyperspectral images, and light detection and ranging (LiDAR) data, either individually or in combination. This does not only allow to derive accurate classification results for many relevant urban classes without preprocessing or feature extraction but also provides insights into which sensor offers the most meaningful data to solve the given classification task. The achieved results on a public benchmark data set are superior to results obtained by deep learning approaches despite being based on only a fraction of training samples.

This work proposes a deep learning based method to de-speckle SAR images that does not require noise-free reference data. Instead, our method exploits the redundancy between images of the same area at different times to train a residual convolutional neural network in a regression framework to predict speckle-free images. Moreover, thanks to end-to-end training of the network, our approach does not require explicit parameter tuning. Experiments show the relevance of our approach on Sentinel 1 images acquired over volcanic areas. The method is shown to compete well with well-known approaches such as the Lee filter and the more recent SAR-BM3D filter.

State of art deep neural network (DNN) models are both memory prohibitive and computationally intensive with millions of connections. Employing these models for an embedded mobile application is resource limited with large amount of power consumption and significant bandwidth requirement (to access the data from the external DRAM). In a custom FPGA hardware the bandwidth access from the DRAM is two to three times higher, compared to the MAC (Multiply-Accumulate) operation. In this paper, we propose a power efficient multi-bit neural network accelerator, where we employ the technique of truncating the partial sum (PSum) results from the previous layer before feeding it into the next layer. We demonstrate that, using our multi-bit accelerator, accuracy is maintained upto bit width of 12. The proposed truncation scheme has 50% power reduction and resource utilization was reduced by 16% for LUTs (Look-up tables), 9% for FFs (Flip-Flops), 19% for BRAMs (Block RAMs) and 7% for Digital Signal Processors (DSPs) when compared with the 32 bits architecture. A large network, AlexNet was used as a benchmark DNN model and Kintex-7 KC705 FPGA was used to test the architecture.

Due to the increasing amount of remotely sensed data, methods for its automatic interpretation become more and more important. Corresponding supervised learning techniques, however, strongly depend on the availability of training data, i.e. data where measurements and labels are provided simultaneously. The creation of reference data for large data sets is very challenging and approaches addressing this task often introduce a significant amount of label noise. While other works focused on the influence of label noise on the training process, this paper studies the impact on the evaluation and shows that the corresponding effects are even more adverse.

The deployment of numerous air- and space-borne remote sensing sensors as well as new data policies led to a tremendous increase of available data. While methods such as neural networks are trained by online or batch processing, i.e. keeping only parts of the data in the memory, other methods such as Random Forests require offline processing, i.e. keeping all data in the memory of the computer. The latter are therefore often trained on a small subset of a larger data set that is hoped to be representative instead of exploiting the information contained in all samples. This paper shows that Random Forests can be trained by batch processing too making their application to large data sets feasible without further constraints. The benefits of this training scheme are illustrated for the use case of land-use classification from PolSAR imagery.

Random Forests (RFs) are a powerful machine learning technique used for various applications including classification, regression, clustering, and manifold learning. The interpretation of a given Random Forest usually relies on statistical values, such as the distribution of path length, leaf impurity, leaf size, etc. All those measures focus on specific aspects and are incapable to provide a holistic understanding of the RF. In this paper, we propose a two-dimensional, easy-to-grasp visualization technique that follows a botanical approach and illustrates several key parameters necessary to understand why a given RF performs in a certain way. The method allows customized mappings of RF characteristics to visual properties and provides the possibility to interactively analyze the forest structure. This allows to determine trees that perform extraordinarily well or bad, to analyze the reasons for their performance, and thus to gain insights into how to change parameter setting to increase performance or efficiency.

While previous work primarily focused on using Tomographic Synthetic Aperture Radar (TomoSAR) data to analyze the 3D structure of the imaged scene, we study its potential for the generation of semantic land cover maps in a supervised framework. We extract different features from the covariance matrices of a tomographic image stack as well as from the tomograms computed by tomographic focusing. To assess the impact of our approach, we compare our results to classification maps obtained from a fully polarimetric image. We show that it is possible to outperform classification results from polarimetric data by carefully designing hand-crafted features which can be extracted either from multi-baseline single polarization covariance matrices or from tomograms obtained after tomographic focusing. Our experiments show a significant gain in the classification accuracy, especially on challenging classes such as heterogeneous city and road.

Random Forests have been shown to able to be applied directly to polarimetric synthetic aperture radar (PolSAR) data instead of to extracted hand-crafted features by adapting the internal node tests. This paper investigates different polarimetric distance measures and their potential to be used by Random Forests for the classification of PolSAR images. The experiments show that using distance measures tailored towards the statistics of PolSAR data outperforms the usage of individual hand-crafted polarimetric features and their combination. However, the differences between accuracies obtained by different suitable distance measures are insignificant allowing to take other aspects into consideration such as computational efficiency.

In this paper, we introduce two spatially adaptive filtering methods to improve the estimation of the covariance matrix (CM), which is required for the processing of tomographic SAR data. We evaluate their effect on scatterer separation and height estimation. We propose several criteria to evaluate such methods and introduce a spatial simulation procedure allowing generating a tomographic image stack from a 3-D building model, assuming a multitrack airborne configuration and a distributed target model incorporating multidimensional speckle. Inversion of such a model requires the estimation of a CM from the data. Consequently, we propose two nonlocal methods to improve the estimation of the CM. The first one was previously introduced for polarimetric data and uses pixel similarities based on Riemannian distances between CMs. The second one is a new method extending the previous one to similarities between patches. We show the importance of spatial adaptivity in covariance estimation by comparing the 3-D reconstructions obtained with our filters and other methods. Further experiments on simulated and L-band experimental data show the ability of the nonlocal filters to improve the height estimation and scatterer separation in layover areas thanks to their smoothing and edge-preserving properties.

In this paper, cell nuclei attributed relational graphs are extensively studied and comparatively analyzed for effective knowledge description and classification in H&E stained whole slide images of gastric cancer. This includes design and implementation of multiple graph variations with diverse tissue component characteristics and architectural properties to obtain enhanced image representations, followed by hierarchical ensemble learning and classification. A detailed comparative analysis of the proposed graph-based methods, also with the established low-level, object-level and high-level image descriptions is performed, that further leads to a hybrid approach combining salient visual information. Quantitative evaluation of investigated methods suggests the suitability of particular graph variants for automatic classification using H&E stained histopathological gastric cancer whole slide images based on HER2 immunohistochemistry.

In this paper, we introduce two spatially adaptive covariance filtering methods and evaluate their effect on scatterer separation and height estimation from tomographic SAR. The first one was previously introduced for polarimetric data and uses pixel similarities based on Riemannian distances between covariance matrices. The second one is a new method extending the previous one to patch-based similarities. We show the importance of spatial adaptivity in covariance estimation by comparing the 3D reconstructions obtained with our nonlocal filters and the boxcar filter. Our experiments on simulated and L-band experimental data show the ability of the non-local filters to improve the height estimation and scatterer separation in layover areas thanks to their smoothing and edge preserving properties.

The automatic generation of semantic maps from remotely sensed imagery by supervised classifiers has seen much effort in the last decades. The major focus has been on the improvement of the interplay between feature operators and classifiers, while experimental design and test data generation has been mostly neglected. This paper shows that sampling strategies applied to partition the available reference data into train and test sets have a large influence on the quality and reliability of the estimated generalization error. It illustrates and discusses problems of common choices for sampling schemes, i.e. the violation of the independence assumption and the illusion of the availability of global knowledge in the training data. Furthermore, a novel sampling strategy is proposed which circumvents these problems and achieves a less biased estimate of the classification error.

TomoSAR (Tomographic SAR) is a technique allowing to extend SAR imaging to the third dimension by using several images of a scene acquired from different sensor positions. 3D point clouds extracted thanks to tomographic processing methods are often corrupted by noise and artifacts which need to be corrected. In this paper, we propose a simple convex optimization formulation that exploits the geometric constraint that the line of sight between a sensor and a surface measurement must be unobscured. We demonstrate the ability of our method to denoise point clouds and fill holes on both synthetic and experimental DLR E-SAR data.

With increasing availability of different sensors for earth observation, data fusion gained more and more importance. While previous publications focussed on new sensor combinations, new fusion techniques, or new applications, this work investigates at which stage of the image analysis pipeline the fusion process is most beneficial. The fusion of an optical and a SAR image for the task of land cover classification serves as an example. The experimental results indicate, that although the fusion of complementary data is generally advantageous, it is most helpful at later stages of the classification process.

One of the major problems of keypoint-based alignment of SAR and optical images is that keypoint operators react to very different object structures in both image types. This leads to a small mutual overlap in the corresponding sets of keypoints. This paper proposes to cast the task of keypoint detection as a classification problem. A machine-learning based classifier is trained to predict whether a SAR image pixel corresponds to a keypoint in the optical image or not. Experimental results indicate, that the mutual overlap of keypoints can be doubled by the proposed approach.

Automatic detection of necrosis in histological images is an interesting problem of digital pathology that needs to be addressed. Determination of presence and extent of necrosis can provide useful information for disease diagnosis and prognosis, and the detected necrotic regions can also be excluded before analyzing the remaining living tissue. This paper describes a novel appearance-based method to detect tumor necrosis in histopathogical whole slide images. Studies are performed on heterogeneous microscopic images of gastric cancer containing tissue regions with variation in malignancy level and stain intensity. Textural image features are extracted from image patches to efficiently represent necrotic appearance in the tissue and machine learning is performed using support vector machines followed by discriminative thresholding for our complex datasets. The classification results are quantitatively evaluated for different image patch sizes using two cross validation approaches namely three-fold and leave one out cross validation, and the best average cross validation rate of 85.31% is achieved for the most suitable patch size. Therefore, the proposed method is a promising tool to detect necrosis in heterogeneous whole slide images, showing its robustness to varying visual appearances.

In this paper, we propose to reconstruct, in 3D, an isolated building. We introduce a treatment chain, from SAR data to 3D rendering. We define the Tomographic Bilateral filter (TomoSAR-BLF) which improve drastically the estimation of the covariance matrix by preserving the edge and the nature of the different object. The rest of the processing consists mainly by generating the tomograms using spectral analysis, extracting point-cloud using the TomoSNI approach and then rendering the 3D building using a mesh obtained by triangulation and surface rendering method.

Random Forests and their many variations developed to one of the most successful instruments to automatically analyse image data. One of the most crucial parts is the definition and selection of node tests within the individual trees, which among other things allow for trade-offs between accuracy and computational load. This paper discusses several different approaches to test creation and compares them based on their classification performance on polarimetric synthetic aperture radar data. The experiments show that selecting the best out of multiple randomly generated node tests leads to the highest accuracy with the smallest computational effort.

In this paper, we introduce an automatic parameter selection technique for the polarimetric bilateral filter. The method is inspired by the theory of unbiased risk estimation that allows to compare different estimators with respect to a loss function. Moreover, a local risk estimation allows spatially varying parameters. We demonstrate our approach on experimental data and show how the method improves the smoothing capabilities of the filter in homogeneous areas while retaining its edge preserving properties.

This paper describes the analysis of complex scenario of urban area by means of a reduced number of tomographic SAR data. First, the tomographic data are processed using the standard MUSIC algorithm, which requires an estimation of the data covariance matrix and an estimation of number of sources. We introduce the tomographic bilateral filter (To-moBLF) to improve the estimation of the covariance matrix and we propose a new model order selection scheme using the tomographic entropy parameters TomoH. Then point of interest are extracted using the Tomographic Signal-to-Noise Index (TomoSNI) algorithm.

Segmentation of Synthetic Aperture Radar (SAR) images is often only understood as the partitioning of the image into rather small regions which are homogeneous with respect to scattering processes. This paper proposes an adaption of the graph-cut image segmentation framework to the unique characteristics of polarimetric SAR images by using a Wishart-distribution based distance measure for local segmentation cues and simple, real-valued features derived from the complex-valued coherency matrix. The proposed method is evaluated on different polarimetric SAR images, for different objects of interest, and with a wide range of parameters. The results show that the proposed framework is able to derive accurate object/non-object segmentations. Best results are obtained for forest areas by usage of a log-transform of the polarimetric intensities.

The paper proposes a novel approach to object classification and segmentation in multi-channel (e.g. polarimetric) SAR data. The classifier is intended for particularly difficult problems, where objects of interest exhibit a high degree of radiometric, polarimetric and geometric heterogeneity, both within individual object instances and across the object category as a whole. Classification is based on a non-parametric characterization of scene contents that avoids model assumptions liable to fail in this scenario. The classifier structure is based on a combination of techniques developed for related problems in computer vision: the cascade architecture helps breaking down the problem into manageable stages while random forests provide a powerful framework for learning and combining discriminative classification rules. In addition, scale space techniques explicitly introduce non-local, contextual and geometric information into the classification process. Preliminary results illustrate the potential of the proposed approach with respect to the task of building segmentation in dual-polarized TerraSAR-X data.

In this paper we introduce a method that allows automatic extraction of planar features from tomographic SAR data. Our approach takes advantage of the spatial connectivity of pixels from tomographic height maps in order to retrieve planar patches from noise corrupted complex scenes. We demonstrate how our method outperforms the well-known RANSAC algorithm over synthetic and experimental data.

In this paper we propose a new speckle filter for polarimetric SAR data based on the bilateral filter. We also study the use of Riemannian metrics related to the Hermitian nature of the complex covariance matrices. The approach is validated over synthetic and experimental data. The method achieves a strong smoothing in homogeneous areas while preserving both polarimetric and spatial information.

In this paper we present a SAR tomographic post-processing technique to extract point of interest from SAR tomograms. We propose a signal-to-noise index adapted to tomogram generated by SP-MUSIC-1 (single polarization, assuming 1 scatterer) that quantifies the separation of scatterer peak from other artifacts (noise,...). This index allows to remove artifacts caused by speckle noise, low SNR, error in orbit position, etc. The domain of validity of this approach is also analyzed over two different regions: isolated buildings and building surrounded with vegetation. Experiment results are shown using a multibaseline dataset acquired in L-band by DLR's experimental SAR (E-SAR) on a test site near Oberpfaffenhofen/Germany.

The development of Geographical Information Systems applications involving fusion of data from different space-borne imaging sensors inevitably requires a preliminary registration of the images. In case of Synthetic Aperture Radar (SAR) and optical sensors, the registration is particularly challenging due to the vast radiometric differences in the data. In this paper, we present a novel method to register SAR and optical images automatically. It provides an accurate registration despite the radiometric differences in the images. Moreover, this paper introduces a Monte Carlo formulation of the image registration problem.

This paper proposes a new method for a measure of coherent similarity between temporal multichannel synthetic aperture radar (SAR) images and its implementation to change detection application. The method is based on mutual information (MI) from information theory. The MI measures the amount of information in common between coherent temporal multichannel SAR acquisitions. In order to develop an algorithm for all kinds of SAR images, such as interferometric SAR, polarimetric-interferometric SAR (PolInSAR), and partial PolInSAR, first, the joint density function of temporal multichannel images based on their second-order statistics has been derived. Then, the derived joint density function is used to calculate an analytical expression for the MI between temporal images, which is assumed to be maximal if the temporal images are identical. Although, in this paper, a new coherent similarity measure has analytically been derived for temporal polarimetric SAR images based on complex Wishart process in time, since the mathematical formulation is general, it can equally well be implemented into any kind of multivariate remote sensing data, such as multispectral optical and interferometric images after small continuation. This derived quantity has been implemented for change detection application whose aim is to characterize the temporal behavior of the acquisitions. A comparison between the proposed and the other well-known change detection methods by means of scene characterization is shown, describing the advantages due to the fact that the proposed change detector involves almost every facet of applied change detection.

This paper proposes a new method for change detection measurement including whole SAR imaging modes such as PolIn- SAR, partial PolInSAR and InSAR in a set of multi-temporal multidimensional SAR images. The method is based on the special case of Kullback-Leibler (KL-divergence) test, known as Mutual Information. In order to develop an algorithm, firstly the joint distribution of PolInSAR data set, based on the second order statistics has been derived. Such a derivation accounts for the whole multi-temporal SAR images. Then the mutual information is used to measure the difference between the joint density of multi-temporal PolSAR data sets and their marginal density known as complex Wishart distribution. A comparison between the proposed and the other well-known change detection (e.g. cross correlation) technique is shown by means of real data, describing the advantages due to the fact that the proposed change detector involves almost every facet of the applied change detection.

Building detection from Synthetic Aperture Radar (SAR) images states a particular important as well as difficult problem. The high-resolution which is necessary to distinguish single buildings as well as the geometric and di-electric properties of dense urban areas cause most assumptions to fail, that are commonly made in SAR data analysis. This paper proposes the usage of Random Forests for building detection from high-resolution Polarimetric Synthetic Aperture Radar (PolSAR) imagery. Random Forests can handle high-dimensional input and therefore a large set of different features, they are known to lead to good classification performance in terms of robustness and accuracy, and are nevertheless seldomly applied to analysis of PolSAR images in general and building detection in particular. This paper presents first results of Random Forests when applied to a building detection task and shows their successful applicability.

In this paper, the accuracy assessment of the recently proposed Maximum Likelihood (ML) texture tracking algorithm is discussed. Its comparison with the well known texture tracking technique, i.e., Normalized Incoherent Cross Correlation (NICC), has also been investigated in the case of the presence of multiplicative noise structure.

Supervised learning algorithms are important methods to automatically interpret image data in general as well as PolSAR data in particular. However, they suffer from the need of a training set, which has to contain manually labelled data. Un-supervised methods do not demand this kind of data, but cannot be directly used to assign user-defined class labels to image regions. This paper proposes a semi-supervised method to overcome both shortcomings. The data is analysed by an un-supervised clustering algorithm under the usage of all available information. Simultaneously each pixel is classified by a supervised method using the information available at the current phase of clustering.

Building extraction from high resolution Synthetic Aperture Radar (SAR) images can benefit from modelling the interaction of several elements in urban scene. This paper proposes a Bayesian approach to exploit the interplay. The appearances of buildings in SAR images are dependent on their orientation angles. We estimate the orientation angles of buildings by supervised learning. The knowledge of other object classes could contribute to the building detection. We extract surface evidence of major object classes. The integration of angle estimation, building detection and surface classes provides promising results.

Scatterers in synthetic aperture radar (SAR) images exhibit high dependence on scatterer-sensor orientations. This phenomenon is prevalent in urban areas. This paper applies hidden Markov model (HMM) to characterize the dependence and model the variations with respect to orientation. Buildings in high resolution SAR images of urban areas are studied. Buildings regions are divided into several discrete classes according to their orientation angles. We model the variations of scatterers characteristics throughout the subapertures using HMM. Subapertures are generated using wavelet packet decomposition. The experimental results show that HMM is efficient in building detection and orientation angle identification. HMMs trained using different feature sets are investigated. The evolution of scatterer states in subapertures are obtained from the HMM inference.

The performance of a tracking algorithm considering remotely sensed data strongly depends on a correct statistical description of the data, i.e., its noise model. The objective of this paper is to introduce a new intensity tracking algorithm for synthetic aperture radar (SAR) data, considering its multiplicative speckle/noise model. The proposed tracking algorithm is discussed regarding the measurement of glacier velocities. Glacier monitoring exhibits complex spatial and temporal dynamics including snowfall, melting, and ice flows at a variety of spatial and temporal scales. Due to these complex characteristics, most traditional methods based on SAR suffer from speckle decorrelation that results in a low signal-to-noise ratio. The proposed tracking technique improves the accuracy of the classical intensity tracking technique by making use of the temporal speckle structure. Even though a new intensity-based matching algorithm is proposed, particularly for incoherent data sets, the analysis of the proposed technique was also performed for correlated data sets. As it is demonstrated, the velocity monitoring can be continuously performed by using the maximum likelihood (ML) texture tracking without any assumption concerning the correlation of the data set. The ML texture tracking approach was tested on ENVISAT-ASAR data acquired during summer 2004 over the Inyltshik glacier in Kyrgyzstan, representing one of the largest alpine glacier systems of the world. It will be demonstrated that the proposed technique is capable of robustly and precisely detecting the surface velocity field and velocity changes in time.

Recent pose invariant methods try to model the subject specific appearance change across pose. For this, however, almost all of the existing methods require a perfect alignment between a gallery and a probe image. In this paper we present a pose invariant face recognition method, centered on modeling joint appearance of gallery and probe images across pose, that do not require the facial landmarks to be detected as such. We propose novel extensions by introducing to use a more robust feature description as opposed to pixel-based appearances. Using such features we put forward to synthesize the non-frontal views to frontal. Furthermore, using local kernel density estimation, instead of commonly used normal density assumption, is suggested to derive the prior models. Our method does not require any strict alignment between gallery and probe images which makes it particularly attractive as compared to the existing state of the art methods. Improved recognition across a wide range of poses has been achieved using these extensions.

This paper presents a preliminary comparison of mean shift segmentation, efficient graph-based segmentation and normalized cuts for segmenting meter-resolution SAR data in urban areas. The small patches generated by these bottom-up segmentation algorithms provide spatial support for object detection. We evaluate their performances on ground truth data by varying their parameters. In order to obtain better spatial support, we apply multiple segmentations to a single segmentation of each algorithm. The multiple segmentations are representative samplings of the entire segmentation space. The experimental results demonstrate that the three algorithms are promising for SAR image segmentation, and that the multiple segmentations improves the abilities of providing spatial support.

Numerous techniques were invented in computer vision and photogrammetry to obtain spatial information from digital images. We intend to describe and improve the performance of these vision techniques by providing test objectives, data, metrics and test protocols. In this paper we propose a comprehensive benchmarking dataset for evaluating a variety of automatic surface reconstruction algorithms (shape-from-X) and a methodology for comparing their results.

In this paper, we present a regularization approach on discrete graph spaces for perceptual image segmentation via semi-supervised learning. In this approach, first, a spectral clustering method is embedded and extended into regularization on discrete graph spaces. In consequence, the spectral graph clustering is optimized and smoothed by integrating top-down and bottom-up processes via semi-supervised learning. Second, a designed nonlinear diffusion filter is used to maintain semi-supervised learning, labeling and differences between foreground or background regions. Furthermore, the spectral segmentation is penalized and adjusted using labeling prior and optimal window-based affinity functions in a regularization framework on discrete graph spaces. Experiments show that the algorithm achieves perceptual and optimal image segmentation. The algorithm is robust in that it can handle images that are formed in variational environments.

A grown importance in long-term operational glacier monitoring has emerged, mainly due to the connection of glacier recession to climate changes. Up to now, mainly two types of methods have been used for the estimation of glacier flow velocities: Image matching and differential interferometry (DInSAR). Although the principal potential of DInSAR for glacier velocity estimation has been shown in several case studies, its successful application is often limited by phase noise, described by the coherence. Additionally, the glacier velocity is often too large to be analysed by means of DInSAR since this method can be too sensitive to correctly track the large displacements occurring during a typical data acquisition interval of one month. SAR amplitude images are not limited by phase stability problems like in DInSAR and can reliably be acquired on a regular basis. In this work, a novel algorithm for computing the velocity field and motion parameters from a sequence of SAR amplitude images are presented. The algorithm is based on the vector relaxation combined with standardized cross- covariance matrix information and cross-correlation techniques. The cross-correlation is used to indicate the candidate motion vectors for each pixel. After this step, by a relaxation operation local smoothness constraints are introduced into the estimated flow pattern, leading to a more homogeneous velocity estimation. In order to handle fast motion and reduce the mismatches, the mentioned algorithms are applied in different scales and linked using anisotropic diffusion equation in case of multiscale cross-correlation. This significantly improves the reliability of the motion detection in the presence of noise, inherent in case of SAR data.

The paper presents a new framework for the classification of polarimetric SAR data. The underlying model introduces cyclic conditional dependencies among the class labels assigned to neighboring observations as a mechanism to regulate the spatial homogeneity of classification results. Classification is posed as an inference problem, and is solved by coherently integrating expectation maximization and graph cut optimization. Results based on real SAR data are presented.

Abstract — Effective feature extraction is the basis of every approach to automated image analysis. An important class of extraction operators, point and region of interest detectors, has not yet been developed for SAR Polarimetry. This paper describes a region of interest operator designed to identify distinctive regions in a scale invariant fashion. The work presented includes a novel definition of image entropy, in the information theoretical sense, for polarimetric SAR image content, as well as a rigorous statistical analysis of the operators scale selection mechanism. This analysis establishes the ability to identify a region irrespective of its size. The results presented include the application of the operator to real data and demonstrations of the operators scale invariance. I.

Presently, a data fusion contest is conducted to compare the potential of airborne SAR with optical sensors for mapping applications. The goal of the test is to answer two questions: (1) Can state-of-the-art airborne SAR compete with optical sensors in the mapping domain? (2) What can be gained when SAR and optical images are used in combination, i.e., when methods for information fusion are applied? The test is organized in the framework of the IEEE GRSS data fusion technical committee (DFC), ISPRS working group III/6 "Multi-Source Vision", which both have strong relations with scientists, active in research on sensor fusion and automation in mapping, and - as the provider of the main organizational framework - the European Organization for Experimental Photogrammetric Research (OEEPE), which is the European research platform of national mapping agencies and other institutions, regarding technology developments to optimize the use of core data in a geoinformation infrastructure context. In the preparatory phase of the test, which has been started on the occasion of IGARSS 2001, test data has been collected and the scope of object extraction for mapping has been defined. The outcome of this phase, i.e. the test imagery to be used in the contest and its potential for mapping is presented in this paper.

We propose a general method to separate topography- and displacement-related phase components in repeat-pass interferograms based on adjustment theory. The method is exemplarily demonstrated for the analysis of rheology of Arctic glaciers. More than two multitemporal interferometric data sets are combined in a least squares adjustment based on a Gauss-Markov model. Within the adjustment algorithm glacier flow is modeled by a polynomial function. This technique on the one hand allows us to improve the separation of topography- and displacement-related phase components and on the other hand provides glaciological information about the flow characteristics of observed glaciers.

Due to the speckle effect of coherent imaging the detection of lines in SAR scenes is considerably move difficult than in optical images. A new approach to detect lines in noisy images using a Markov random field (MRF) model and Bayesian classification is proposed. The unobservable object classes of single pixels are assumed to fulfil the Markov condition, i.e. to depend on the object classes of neighboring pixels only. The influence of neighboring line pixels is formulated based on potentials derived from a random walk model. Locally, the image data is evaluated with a rotating template. As SAR intensity data is deteriorated by multiplicative noise, the response of the local line detector is a normalized intensity ratio which results in a constant false alarm rate. The approach integrates intensity, coherence from interferometric processing of a SAR scene pair, and given Geographic Information System (GIS) data.