Large Language Models (LLMs) enable a new ecosystem with many downstream applications, called LLM applications, with different natural language processing tasks. The functionality and performance of an LLM application highly depend on its system prompt, which instructs the backend LLM on what task to perform. Therefore, an LLM application developer often keeps a system prompt confidential to protect its intellectual property. As a result, a natural attack, called prompt leaking, is to steal the system prompt from an LLM application, which compromises the developer's intellectual property. Existing prompt leaking attacks primarily rely on manually crafted queries, and thus achieve limited effectiveness.

Designing machine learning algorithms that are accurate yet fair, not discriminating based on any sensitive attribute, is of paramount importance for society to accept AI for critical applications. In this article, we propose a novel fair representation learning method termed the Rényi Fair Information Bottleneck Method (RFIB) which incorporates constraints for utility, fairness, and compactness (compression) of representation, and apply it to image and tabular data classification. A key attribute of our approach is that we consider - in contrast to most prior work - both demographic parity and equalized odds as fairness constraints, allowing for a more nuanced satisfaction of both criteria. Leveraging a variational approach, we show that our objectives yield a loss function involving classical Information Bottleneck (IB) measures and establish an upper bound in terms of two Rényi measures of order$ \boldsymbol {\alpha }$on the mutual information IB term measuring compactness between the input and its encoded embedding. We study the influence of the$ \boldsymbol {\alpha }$parameter as well as two other tunable IB parameters on achieving utility/fairness trade-off goals, and show that the$ \boldsymbol {\alpha }$parameter gives an additional degree of freedom that can be used to control the compactness of the representation. Experimenting on three different image datasets (EyePACS, CelebA, and FairFace) and two tabular datasets (Adult and COMPAS), using both binary and categorical sensitive attributes, we show that on various utility, fairness, and compound utility/fairness metrics RFIB outperforms current state-of-the-art approaches.

Membership inference (MI) attacks are more diverse in a Federated Learning (FL) setting, because an adversary may be either an FL client, a server, or an external attacker. Existing defenses against MI attacks rely on perturbations to either the model's output predictions or the training process. However, output perturbations are ineffective in an FL setting, because a malicious server can access the model without output perturbation while training perturbations struggle to achieve a good utility. This paper proposes a novel defense, called CIP, to fortify FL against MI attacks via a client-level input perturbation during training and inference procedures. The key insight is to shift each client's local data distribution via a personalized perturbation to get a shifted model. CIP achieves a good balance between privacy and utility. Our evaluation shows that CIP causes accuracy to drop at most 0.7% while reducing attacks to random guessing.

In contrast to perturbation-based attacks, patch-based attacks are physically realizable, and are therefore increasingly studied. However, prior work neglects the possibility of adaptive attacks optimized for 3D pose. For the first time, to our knowledge, we consider the challenge of designing and evaluating attacks on image sequences using 3D optimization along entire 3D kinematic trajectories. In this context, we study a type of dynamic attack, referred to as "adaptive active attacks" (AAA), that takes into consideration the pose of the observer being targeted. To better address the threat and risk posed by AAA attacks, we develop several novel risk-based and trajectory-based metrics. These are designed to capture the risk of attack success for attacking earlier in the trajectory to derail autonomous driving systems as well as tradeoffs that may arise given the possibility of additional detection. We evaluate performance of white-box targeted attacks using a subset of ImageNet classes, and demonstrate, in aggregate, that AAA attacks can pose threats beyond static attacks in kinematic settings in situations of predominantly looming motion (i. e., a prevalent use case in automated vehicular navigation). Results demonstrate that AAA attacks can exhibit targeted attack success exceeding 10% in aggregate, and for some specific classes, up to 15% over their static counterparts. However, taking into consideration the probability of detection by the defender shows a more nuanced risk pattern. These new insights are important for guiding future adversarial machine learning studies and suggest researchers should consider defense against novel threats posed by dynamic attacks for full trajectories and videos.

Whilst adversarial attack detection has received considerable attention, it remains a fundamentally challenging problem from two perspectives. First, while threat models can be well-defined, attacker strategies may still vary widely within those constraints. Therefore, detection should be considered as an open-set problem, standing in contrast to most current detection approaches. These methods take a closed-set view and train binary detectors, thus biasing detection toward attacks seen during detector training. Second, limited information is available at test time and typically confounded by nuisance factors including the label and underlying content of the image. We address these challenges via a novel strategy based on random sub-space analysis. We present a technique that utilizes properties of random projections to characterize the behavior of clean and adversarial examples across a diverse set of subspaces. The self-consistency (or inconsistency) of model activations is leveraged to discern clean from adversarial examples. Performance evaluations demonstrate that our technique (AUC ∈ [0.92, 0.98]) outperforms competing detection strategies (AUC ∈ [0.30, 0.79]), while remaining truly agnostic to the attack strategy (for both targeted/untargeted attacks). It also requires significantly less calibration data (composed only of clean examples) than competing approaches to achieve this performance.

We propose a novel method for enforcing AI fairness with respect to protected or sensitive factors. This method uses a dual strategy performing training and representation alteration (TARA) for the mitigation of prominent causes of AI bias. It includes the use of representation learning alteration via adversarial independence to suppress the bias-inducing dependence of the data representation from protected factors and training set alteration via intelligent augmentation to address bias-causing data imbalance by using generative models that allow the fine control of sensitive factors related to underrepresented populations via domain adaptation and latent space manipulation. When testing our methods on image analytics, experiments demonstrate that TARA significantly or fully debiases baseline models while outperforming competing debiasing methods that have the same amount of information-for example, with (% overall accuracy, % accuracy gap) = (78.8, 0.5) versus the baseline method's score of (71.8, 10.5) for Eye-PACS, and (73.7, 11.8) versus (69.1, 21.7) for CelebA. Furthermore, recognizing certain limitations in current metrics used for assessing debiasing performance, we propose novel conjunctive debiasing metrics. Our experiments also demonstrate the ability of these novel metrics in assessing the Pareto efficiency of the proposed methods.

We investigate the use of parameterized families of information-theoretic measures to generalize the loss functions of generative adversarial networks (GANs) with the objective of improving performance. A new generator loss function, least kth-order GAN (LkGAN), is introduced, generalizing the least squares GANs (LSGANs) by using a kth-order absolute error distortion measure with k≥1 (which recovers the LSGAN loss function when k=2). It is shown that minimizing this generalized loss function under an (unconstrained) optimal discriminator is equivalent to minimizing the kth-order Pearson-Vajda divergence. Another novel GAN generator loss function is next proposed in terms of Rényi cross-entropy functionals with order α>0, α≠1. It is demonstrated that this Rényi-centric generalized loss function, which provably reduces to the original GAN loss function as α→1, preserves the equilibrium point satisfied by the original GAN based on the Jensen-Rényi divergence, a natural extension of the Jensen-Shannon divergence. Experimental results indicate that the proposed loss functions, applied to the MNIST and CelebA data sets, under both DCGAN and StyleGAN architectures, confer performance benefits by virtue of the extra degrees of freedom provided by the parameters k and α, respectively. More specifically, experiments show improvements with regard to the quality of the generated images as measured by the Fréchet inception distance score and training stability. While it was applied to GANs in this study, the proposed approach is generic and can be used in other applications of information theory to deep learning, for example, the issues of fairness or privacy in artificial intelligence.

Our work focuses on unsupervised and generative methods that address the following goals: (1) learning unsupervised generative representations that discover latent factors controlling image semantic attributes, (2) studying how this ability to control attributes formally relates to the issue of latent factor disentanglement, clarifying related but dissimilar concepts that had been confounded in the past, and (3) developing anomaly detection methods that leverage representations learned in the first goal. For goal 1, we propose a network architecture that exploits the combination of multiscale generative models with mutual information (MI) maximization. For goal 2, we derive an analytical result, lemma 1, that brings clarity to two related but distinct concepts: the ability of generative networks to control semantic attributes of images they generate, resulting from MI maximization, and the ability to disentangle latent space representations, obtained via total correlation minimization. More specifically, we demonstrate that maximizing semantic attribute control encourages disentanglement of latent factors. Using lemma 1 and adopting MI in our loss function, we then show empirically that for image generation tasks, the proposed approach exhibits superior performance as measured in the quality and disentanglement of the generated images when compared to other state-of-the-art methods, with quality assessed via the Fréchet inception distance (FID) and disentanglement via mutual information gap. For goal 3, we design several systems for anomaly detection exploiting representations learned in goal 1 and demonstrate their performance benefits when compared to state-of-the-art generative and discriminative algorithms. Our contributions in representation learning have potential applications in addressing other important problems in computer vision, such as bias and privacy in AI.

We develop a framework for novelty detection (ND) methods relying on deep embeddings, either discriminative or generative, and also propose a novel framework for assessing their performance. While much progress was made recently in these approaches, it has been accompanied by certain limitations: most methods were tested on relatively simple problems (low resolution images / small number of classes) or involved non-public data; comparative performance has often proven inconclusive because of lacking statistical significance; and evaluation has generally been done on non-canonical problem sets of differing complexity, making apples-to-apples comparative performance evaluation difficult. This has led to a relative confusing state of affairs. We address these challenges via the following contributions: We make a proposal for a novel framework to measure the performance of novelty detection methods using a trade-space demonstrating performance (measured by ROCAUC) as a function of problem complexity. We also make several proposals to formally characterize problem complexity. We conduct experiments with problems of higher complexity (higher image resolution / number of classes). To this end we design several canonical datasets built from CIFAR-10 and ImageNet (IN-125) which we make available to perform future benchmarks for novelty detection as well as other related tasks including semantic zero/adaptive shot and unsupervised learning. Finally, we demonstrate, as one of the methods in our ND framework, a generative novelty detection method whose performance exceeds that of all recent best-in-class generative ND methods.

Efficient exploration through unknown environments remains a challenging problem for robotic systems. In these situations, the robot's ability to reason about its future motion is often severely limited by sensor field of view (FOV). By contrast, biological systems routinely make decisions by taking into consideration what might exist beyond their FOV based on prior experience. We present an approach for predicting occupancy map representations of sensor data for future robot motions using deep neural networks. We develop a custom loss function used to make accurate prediction while emphasizing physical boundaries. We further study extensions to our neural network architecture to account for uncertainty and ambiguity inherent in mapping and exploration. Finally, we demonstrate a combined map prediction and information-theoretic exploration strategy using the variance of the generated hypotheses as the heuristic for efficient exploration of unknown environments.

This work studies joint camera and robotic manipulator control for reaching tasks in complex environments with obstacles and occluders. We obviate the conventional challenges involved in complex perception, planning, and control modules and careful calibration for sensing and actuation and seek a solution leveraging deep reinforcement learning (DRL). Our method using DRL and deep Q-learning learns a policy for robot actuation and perception control, mapping directly raw image pixels inputs into camera motion and manipulator joint control actions outputs. We show results comparing different training approaches, and demonstrating competency for increasingly complex situations and degrees of freedom. These preliminary experiments suggest the effectiveness and robustness of the proposed approach.

This work investigates a hybrid method based on random forests and deep image features to combine non-visual side channel information with image data for classification. We apply this to automated retinal image analysis (ARIA) and the detection of age-related macular degeneration (AMD). For evaluation, we use a dataset collected by the National Institute of Health with over 4000 study participants. The non-visual side channel data includes information related to demographics (e.g. ethnicity), lifestyle (e.g. sunlight exposure), and prior conditions (e.g. cataracts). Our study, which compares the performance of different feature combinations, offers preliminary results that constitute a baseline for future investigations on joint deep visual and side channel feature exploitation for AMD detection. This approach could potentially be used for other medical image analysis problems.

This study addresses the development of machine learning methods for reduced space ultrasound to perform automated prescreening of breast cancer. The use of ultrasound in low-resource settings is constrained by lack of trained personnel and equipment costs, and motivates the need for automated, low-cost diagnostic tools. We hypothesize a solution to this problem is the use of 1D ultrasound (single piezoelectric element). We leverage random forest classifiers to classify 1D samples of various types of tissue phantoms simulating cancerous, benign lesions, and non-cancerous tissues. In addition, we investigate the optimal ultrasound power and frequency parameters to maximize performance. We show preliminary results on 2-, 3- and 5-class classification problems for the ideal power/frequency combination. These results demonstrate promise towards the use of a single-element ultrasound device to screen for breast cancer.

This study focuses on using ultrasound (US) biomarkers for characterizing myopathies and in particular myositis. US offers an opportunity to deliver diagnostics in clinical settings at a fraction of the cost and discomfort entailed in current workflows. US is also better suited for usage in under-resourced environments. This paper is focused on studying the link between biomarkers related to absolute and relative echo intensity of muscle tissue and the presence and severity of myositis disease. We show that there is good correlation between these biomarkers and the severity of muscle disease rated by the Heckmatt criteria. A moderate correlation is also found between these biomarkers and muscles categorized by healthy vs. diseased status of each patient. Experimental data involving 37 patients (9 polymyositis, 3 dermatomyositis, 9 inclusion body myositis, and 16 healthy patients) and seven muscle groups show correlations up to 0.91.

Deep convolutional neural networks (DCNNs) perform on par or better than humans for image classification. Hence efforts have now shifted to more challenging tasks such as object detection and classification in images, video or RGBD. Recently developed region CNNs (R-CNN) such as Fast R-CNN [7] address this detection task for images. Instead, this paper is concerned with video and also focuses on resource-limited systems. Newly proposed methods accelerate R-CNN by sharing convolutional layers for proposal generation, location regression and labeling [12][13][19][25]. These approaches when applied to video are stateless: they process each image individually. This suggests an alternate route: to make R-CNN stateful and exploit temporal consistency. We extend Fast R-CNNs by making it employ recursive Bayesian filtering and perform proposal propagation and reuse. We couple multi-target proposal/detection tracking (MTT) with R-CNN and do detection-to-track association. We call this approach MRCNN as short for MTT + R-CNN. In MRCNN, region proposals that are vetted via classification and regression in R-CNNs - are treated as observations in MTT and propagated using assumed kinematics. Actual proposal generation (e.g. via Selective Search) need only be performed sporadically and/or periodically and is replaced at all other times by MTT proposal predictions. Preliminary results show that MRCNNs can economize on both proposal and classification computations, and can yield up to a 10 to 30 factor decrease in number of proposals generated, about one order of magnitude proposal computation time savings and nearly one order magnitude improvement in overall computational time savings, for comparable localization and classification performance. This method can additionally be beneficial for false alarm abatement.

We study the problem of classifying images when no training exemplars are available for some image classes, and therefore direct classification is not possible. We use instead semantic attributes: if attributes of yet unseen classes can be determined, then class labels may themselves be decided based on prior knowledge of class to attributes relationships. We present several methods for determining attributes, including (A) an approach based on attribute classifiers, and approaches using (B) MAP and (C) MMSE attribute estimators using image classifiers for known classes. Preliminary tests obtained using a dataset comprised of ImageNet images and Human218 attributes yield encouraging performance.

Myocardial strain is important to assess cardiac function and diagnose cardiovascular disease. Despite the adoption of 4D (volume + time) echocardiography for diagnostic and therapeutic purposes, current clinical practice often relies exclusively on 2D measurements of strain or flow information resulting from Doppler echography. However, strain is a 3D measure of deformation in the radial, circumferential and longitudinal directions and therefore full 3D strain, and in particular out-of- sagittal plane strain components, include important information for diagnostic purposes since they provide additional information on the manner in which the heart lengthens and contracts during diastole and systole. In our prior work, we have developed robust variational optical flow methods to estimate dense myocardial motion. In this study, we extend this methodology to track ventricular outlines, which are subsequently used to compute displacement and deformation fields. This in turn is used to compute volumetric estimates of strain. We test our methods on a dataset of 4D ultrasound acquired in vivo from seven patients, and find good agreement with physiological precepts.

The WHO estimates three million new cases of skin cancer each year. Therefore, there exists a need for prescreening tools that can estimate the biological parameters of human skin, as they can help detect cancers before metastasis. In this paper, we present a novel inverse modeling technique based on Kubelka-Munk theory and machine learning to estimate biological skin parameters from in vivo hyperspec-tral imaging. We use the k-nearest neighbors (k-NN) algorithm in order to estimate skin parameters from their hy-perspectral signatures. We test our methods on 241 hyper-spectral signatures obtained from both genders and three ethnicities, and find encouraging results.

Support Vector Data Description (SVDD) methods have been successfully applied to hyperspectral anomaly detection. Unfortunately, the performance of SVDD methods suffers when noisy or non-informative bands are present in the data. If a set of sparse features could be identified for these techniques, the resulting data may improve SVDD performance while enjoying the benefits of decreased processing overhead. Although band selection has been investigated in previous efforts, this work builds on recent research that has resulted in the development of a theoretical framework for signal classification with sparse representation using L1measures.

Particle filters (PFs) are Bayesian filters capable of modeling nonlinear, non-Gaussian, and nonstationary dynamical systems. Recent research in PFs has investigated ways to appropriately sample from the posterior distribution, maintain multiple hypotheses, and alleviate computational costs while preserving tracking accuracy. To address these issues, a novel utilization of the support vector data description (SVDD) density estimation method within the particle filtering framework is presented. The SVDD density estimate can be integrated into a wide range of PFs to realize several benefits. It yields a sparse representation of the posterior density that reduces the computational complexity of the PF. The proposed approach also provides an analytical expression for the posterior distribution that can be used to identify its modes for maintaining multiple hypotheses and computing the MAP estimate, and to directly sample from the posterior. We present several experiments that demonstrate the advantages of incorporating a sparse kernel density estimate in a particle filter.

Our work addresses pose estimation in a distributed camera framework. We examine how processing cameras can best reach a consensus about the pose of an object when they are each given a model of the object, defined by a set of point coordinates in the object frame of reference. The cameras can only see a subset of the object feature points in the midst of background clutter points, not knowing which image points match with which object points, nor which points are object points or background points. The cameras individually recover a prediction of the object's pose using their knowledge of the model, and then exchange information with their neighbors, performing consensus updates locally to obtain a single estimate consistent across all cameras, without requiring a common centralized processor. Our main contributions are: 1) we present a novel algorithm performing consensus updates in 3-D world coordinates penalized by a 3-D model, and 2) we perform a thorough comparison of our method with other current consensus methods. Our method is consistently the most accurate, and we confirm that the existing consensus method based upon calculating the Karcher mean of rotations is also reliable and fast. Experiments on simulated and real imagery are reported.

We present a method for fast anomaly detection in hyperspectral imagery (HSI) based on the support vector data description (SVDD) algorithm. The SVDD is a single class, non-parametric approach for modeling the support of a distribution. A global SVDD anomaly detector is developed that utilizes the SVDD to model the distribution of the spectra of pixels randomly selected from the entire image. Experiments on wide area airborne mine detection (WAAMD) hyperspectral data show improved receiver operating characteristic (ROC) detection performance when compared to the local SVDD detector and other standard anomaly detectors (including RX and GMRF). Furthermore, one-second processing time using desktop computers on several 256 times 256 times 145 datacubes is achieved.

A support vector algorithm for detecting endmembers in a hyperspectral image is introduced. It is a novel method for finding the spectral convexities in a high-dimensional space which addresses several limitations of previous endmember methods. A new approach for estimating the number of endmembers using rate-distortion theory is also presented. It is based upon the observation that the endmembers form a set of basis vectors for the hyperspectral datacube using the linear mixture model. The result is a fully-automatic method for endmember detection. Experimental results using the Cuprite datacube are presented.

A critical step for fitting a linear mixing model to hyperspectral imagery is the estimation of the abundances. The abundances are the percentage of each end member within a given pixel; therefore, they should be non-negative and sum to one. With the advent of kernel based algorithms for hyperspectral imagery, kernel based abundance estimates have become necessary. This paper presents such an algorithm that estimates the abundances in the kernel feature space while maintaining the non-negativity and sum-to-one constraints. The usefulness of the algorithm is shown using the AVIRIS Cuprite, Nevada image.

This paper presents a method for anomaly detection in hyperspectral images based on the support vector data description (SVDD), a kernel method for modeling the support of a distribution. Conventional anomaly-detection algorithms are based upon the popular Reed-Xiaoli detector. However, these algorithms typically suffer from large numbers of false alarms due to the assumptions that the local background is Gaussian and homogeneous. In practice, these assumptions are often violated, especially when the neighborhood of a pixel contains multiple types of terrain. To remove these assumptions, a novel anomaly detector that incorporates a nonparametric background model based on the SVDD is derived. Expanding on prior SVDD work, a geometric interpretation of the SVDD is used to propose a decision rule that utilizes a new test statistic and shares some of the properties of constant false-alarm rate detectors. Using receiver operating characteristic curves, the authors report results that demonstrate the improved performance and reduction in the false-alarm rate when using the SVDD-based detector on wide-area airborne mine detection (WAAMD) and hyperspectral digital imagery collection experiment (HYDICE) imagery

The paper describes a scheme for detecting vehicles in images. The proposed method approximately models the unknown distribution of the images of vehicles by learning higher order statistics (HOS) information of the 'vehicle class' from sample images. Given a test image, statistical information about the background is learnt 'on the fly'. An HOS-based decision measure then classifies test patterns as vehicles or otherwise. When tested on real images of aerial views of vehicular activity, the method gives good results even on complicated scenes. It does not require any a priori information about the site. However, it is amenable to augmentation with contextual information. The method can serve as an important step towards building an automated roadway monitoring system.

The paper describes a scheme for detecting and tracking people in images. The method effectively combines statistical information about the class of people with motion information for classification and tracking. In this scheme, the unknown distribution of the images of people is approximately modeled by learning higher order statistics (HOS) information of the "people class" from sample images. Given a test image, statistical information about the background is learnt dynamically. A motion detector identifies regions of activity in the image sequence. A classifier based on an HOS-based closeness measure then determines which of the moving objects actually correspond to people in motion. The tracking module uses position information and an HOS-based difference measurement vector to establish correspondence. When tested on real video data with a cluttered background, the performance of the method is found to be quite good. The method can also detect people in static imagery.

An unequal error protection approach for the reliable communication of visual information over additive white Gaussian noise channels is proposed and studied. This method relies on a bandwidth-efficient coded modulation scheme that employs selective channel coding and transmission energy allocation in conjunction with sequence maximum a posteriori soft-decision detection. Experimental results indicate that this scheme exhibits graceful performance degradation as the channel conditions deteriorate and provides substantial objective and subjective improvements over uncoded and equal-error protection systems. Coding gains of up to 4 dB in E/sub b//N/sub o/ are achieved.

We propose a segmentation method based on Polya's (1931) urn model for contagious phenomena. A preliminary segmentation yields the initial composition of an urn representing the pixel. The resulting urns are then subjected to a modified urn sampling scheme mimicking the development of an infection to yield a segmentation of the image into homogeneous regions. This process is implemented using contagion urn processes and generalizes Polya's scheme by allowing spatial interactions. The composition of the urns is iteratively updated by assuming a spatial Markovian relationship between neighboring pixel labels. The asymptotic behavior of this process is examined and comparisons with simulated annealing and relaxation labeling are presented. Examples of the application of this scheme to the segmentation of synthetic texture images, ultra-wideband synthetic aperture radar (UWB SAR) images and magnetic resonance images (MRI) are provided.

Detecting targets occluded by foliage in foliage-penetrating (FOPEN) ultra-wideband synthetic aperture radar (UWB SAR) images is an important and challenging problem. Given the different nature of target returns in foliage and nonfoliage regions and very low signal-to-clutter ratio in UWB imagery, conventional detection algorithms fail to yield robust target detection results. A new target detection algorithm is proposed that (1) incorporates symmetric alpha-stable (SalphaS) distributions for accurate clutter modeling, (2) constructs a two-dimensional (2-D) site model for deriving local context, and (3) exploits the site model for region-adaptive target detection. Theoretical and empirical evidence is given to support the use of the SalphaS model for image segmentation and constant false alarm rate (CFAR) detection. Results of our algorithm on real FOPEN images collected by the Army Research Laboratory are provided.

We propose a segmentation method based on Polya's urn model for contagious phenomena. Initial labeling of the pixel is obtained using a Maximum Likelihood (ML) estimate or the Nearest Mean Classifier (NMC), which are used to determine the initial composition of an urn representing the pixel. The resulting urns are then subjected to a modified urn sampling scheme mimicking the development of an infection to yield a segmentation of the image into homogeneous regions. Examples of the application of this scheme to the segmentation of synthetic texture images, Ultra-Wideband Synthetic Aperture Radar (UWB SAR) images and Magnetic Resonance Images (MRI) are provided.

Target detection in foliage-penetrating (FOPEN), ultrawideband synthetic aperture radar (UWB SAR) images is a challenging problem. Given the low signal-to-clutter ratio, conventional detection algorithms perform poorly in FOPEN SAR images. Using symmetric alpha-stable (S/spl alpha/S) densities to model the impulsive noise, we have developed a region-adaptive automatic target detection (ATD) algorithm. The image is first segmented, and the resulting labeled image is exploited by a region-adaptive target detection algorithm. We evaluate the performance of the algorithm in different frequency bands, and determine which subbands are useful for image segmentation and target detection.

This correspondence addresses the use of a joint source-channel coding strategy for enhancing the error resilience of images transmitted over a binary channel with additive Markov noise. In this scheme, inherent or residual (after source coding) image redundancy is exploited at the receiver via a maximum a posteriori (MAP) channel detector. This detector, which is optimal in terms of minimizing the probability of error, also exploits the larger capacity of the channel with memory as opposed to the interleaved (memoryless) channel. We first consider MAP channel decoding of uncompressed two-tone and bit-plane encoded grey-level images. Next, we propose a scheme relying on unequal error protection and MAP detection for transmitting grey-level images compressed using the discrete cosine transform (DCT), zonal coding, and quantization. Experimental results demonstrate that for various overall (source and channel) operational rates, significant performance improvements can be achieved over interleaved systems that do not incorporate image redundancy.

Robustness as well as the ability to work in an unsupervised mode are two desirable features of algorithms employed on large image databases. This paper describes parameter optimization strategies for such algorithms and motivates these strategies by focussing on aerial image exploitation and studying certain specific aerial image understanding algorithms, namely local vehicle detection and global vehicle configuration detection. The paper first gives a brief introduction to the problem in the context of aerial imagery. Next, a high level description of the algorithms and parameters that need to be optimized is given. Strategies for parameter optimization are illustrated using examples. Finally a discussion on the applicability and scope for improvement of the strategies is given.

We propose a novel video coding scheme to improve the performance of established block-based motion compensation codecs such as MPEG, H261, and H263. The proposed method is a hybrid scheme which introduces model-based global motion compensation as a pre-processing step to the basic block-based motion compensation technique. Performance evaluation tests show that the new method is capable of achieving higher compression rates with the addition of a very small overhead due to global motion estimation. In this paper we describe a codec based on the hybrid motion compensation technique in the context of H263, and present experimental results, comparing the performance of different codecs using rate-distortion curves for several test sequences.

This paper proposes an adaptive source-channel subband coding scheme for the transmission of video over fading wireless channels. A three-dimensional subband decomposition followed by vector quantization of the subband coefficients forms the source coding strategy. For transmission over the channel, the individual subbands are offered different amounts of protection depending on their importance in reconstruction at the receiver. For each subband, the source coding rate as well as the level of protection (quantified by the channel coding rate) are jointly chosen to minimize the total mean-squared distortion suffered by the video coder. The choice of source and channel coding rates depends on the state of the physical channel. We use a finite state model for the fading channel, where every state corresponds to an AWGN channel. This results in a joint source-channel coding scheme that adapts in an optimal way to the current state of a fading channel.

Modern image exploitation tasks have evolved from the early single-image, pixel-based and model-less methods to the current multi-image, multisensor, multiplatform, and model-based approaches. In this context, image positioning, which is the process of establishing the precise geometric relationship of an acquired image to the three-dimensional (3-D) world, has become an enabling technique for state-of-the-art multisensor data exploitation. Precise image positioning provides several benefits. Image registration, traditionally formulated as an image-to-image alignment problem, can now be carried out in accordance with interior and exterior sensor geometries. Images from sensors in arbitrary locations and orientations can be positioned with respect to a focal vertical and geocentric coordinate systems. This paper presents techniques for positioning images derived from various sensors such as electro-optical (E-O), synthetic aperture radar (SAR), and interferometric synthetic aperture radar (IFSAR). Applications to model-supported image exploitation are also discussed.

This paper describes an attentional mechanism based on the interpretation of spectral signatures for detecting regular object configurations in areas of an image delineated using context information. The proposed global operator relies on the spectral analysis of edge structure and exploits spatial as well as frequency-domain constraints derived from known geometrical models of monitored objects. A decision theoretic method for learning acceptance detection regions is presented. Applications of this attentional mechanism are demonstrated for several aerial image interpretation tasks for attentional as well as recognition purposes. Specific examples are described for detecting vehicle formations (such as convoys), qualifying the geometry of detected formations, or monitoring the occupancy of regions of interest (such as parking areas, roads, or open areas). Experiments and sensitivity analysis results are reported.

This paper describes an attentional mechanism based on the interpretation of spectral signatures for detecting regular object configurations in areas of an image delineated using context information. The proposed global operator relies on the spectral analyse's of edge structure and exploits spatial as well as frequency domain constraints derived from known geometrical models of monitored objects. A decision theoretic method for learning decision regions is presented. Applications of this mechanism are demonstrated for several aerial image interpretation tasks. Specific examples are described for detecting vehicle formations (such as convoys), qualifying the geometry of detected formations, or monitoring the occupancy of regions of interest (such as parking areas, roads, or open areas). Experiments and sensitivity analysis results are reported.

Image-to-site model registration is critical to model supported exploitation of aerial and satellite imagery. This paper presents a fully-automatic registration method. This method uses a multi-resolution image-to-image registration process assuming both affine and projective transformations to determine and refine the locations of 3D control points in the new image. Camera resection is subsequently accomplished. Registration results obtained on real imagery show good performance.

A joint source-channel coding technique is proposed for transmitting grey-level images over a binary channel with additive Markov noise. In this scheme, inherent or residual (after source coding) image redundancy is exploited at the receiver in an appropriately designed MAP detector. Two methods are presented. The first method relies on MAP decoding of uncompressed bit-plane encoded images. The second method deals with compressed images (DCT coded and quantized) and uses unequal error protection along with the MAP detection procedure. Experimental results demonstrate that particularly during bad channel conditions, significant performance improvements can be achieved.

We study the performance associated with model-based video coding schemes using global motion models for motion compensation. Reference frames and compensated frame differences are coded using a method similar to MPEG, employing transform coding, quantization, and entropy coding. The traditional block matching motion compensation approach is compared to global motion compensation approaches derived from 3D motion stabilization methods, using similarity and projective transformations. 3D model-based motion compensation is achieved by derotating the input sequence. This leads to a projective transformation, which under certain circumstances, is well approximated by a similarity transformation. Experiments are carried out to analyze the performance of each scheme for comparable coding rates.

This paper addresses the use of spatio-temporal transform methods applied to the analysis of dynamic image sequences and the characterization of image motion. The image motion including a divergent component (resulting from a looming camera component) is analyzed in the spatio-temporal Mellin transform (MT) domain, resulting in the separation of the spectrum into two parts: a structural term corresponding to the spatial MT of the static image, and a kinematic term depending on time-to-collision (a motion support). We examine potential applications of this property for the recovery of image motion from integral image brightness measurements and the computation of time-to-collision using spatio-temporal MT analysis.

We introduce a novel class of nonlinear stochastic filters based on contagion urn schemes. These filters which rely on biologically inspired sampling processes, offer good restoration results on heavily corrupted binary images.

Image stabilization is a key preprocessing step in dynamic image analysis and deals with the removal of unwanted image motion in a video sequence. This paper presents an integrated algorithm for the problem of image stabilization. The algorithm combines various visual cues such as points and horizon lines, and relies on an extended Kalman filter for the estimation of parameters of interest. We study both calibrated and uncalibrated stabilization cases, and consider the problem of the selection of model dynamics for the estimation of warping parameters. Experimental results from video sequences generated from off-road vehicle platforms show good performance of stabilization algorithm.

Situations involving navigation among manoeuvring agents are critical for the study of visual guidance of autonomous vehicles. This paper addresses the case of translational motion with polynomial regimes and defines a general class of temporal parameters (TP) relevant for navigation, enabling a qualitative description of the observed agents' depth trajectories. These parameters are shown to be visually recoverable. Instances of such temporal parameters include Time-to-Collision (TTC) and Time-to-Synchronization (TTS), useful for docking or platooning maneuvers. The results are specialized to lower order motions. The recovery of TTC and TTS for arbitrary regimes is a special corollary of our analysis. Computations from direct and feature-based methods are described. A scheme for addressing model order determination, collision detection and temporal parameter estimation as proposed and tested. Experimental results on synthetic and real images are given.>

This paper addresses the use of transform methods applied to the analysis of dynamic sequences and to the characterization of image motion. It is shown that image motion resulting from arbitrary 3D camera translation is conveniently analyzed in the Mellin transform (MT) domain associated with space as well as time dimensions, resulting in the desired factorization of the generalized spectrum into a structural component corresponding to the spatial MT of the static image and a MT component depending on the image motion itself (a motion support). This has potential applications for the recovery of image motion and time to collision (TTC) from integral image brightness measurements. We study the relationship between TTC to an imaged object on the resulting MT spectral motion support; conversely we study the recovery of TTC from MT spectral analysis along time and space directions. Different cases of Mellin parameters are examined.>

The trajectory planning problem for mobile robots in unknown dynamic workspaces is posed as an optimization problem with optimality criteria the probability of not colliding with the obstacles and the probability of accessing an operational position with respect to a moving target object. The authors study a formal computational framework in which such probabilities can be derived for elementary robot displacements.>

The authors describe a computational framework in which a probabilistic method for noisy sensor-based robotic navigation in dynamic environments can be devised. The aim of the method is to generate an optimal trajectory by considering as optimality criteria the probability of not colliding with the obstacles and the probability of accessing an operational position with respect to a moving target object. A formal framework in which the probability of collision associated with an elementary robot displacement can be calculated is discussed. Estimates on the obstacle kinematic parameters and measures of confidence on these estimates are used to produce the probability of collision associated with any robot displacement. The probability of collision is derived in two steps: a stochastic model is defined in the kinematic state space of the obstacles and collision events are given a simple geometric characterization in this state space.>