State prediction from noisy time series measurements is a challenging task found in areas like intelligent transport, structural health monitoring, and environmental monitoring. This paper proposes a Spatio-Temporal Deep Kernel Gaussian Process (STDK GP) approach, which leverages the feature extraction capabilities of convolutional neural networks with the uncertainty quantification of Gaussian Process (GP) methods. The model features a composite spatio-temporal kernel that operates on learned deep features. A key aspect of our approach is that this kernel is learned end-to-end with the feature extractor, allowing it to effectively capture complex spatial and temporal patterns and enabling robust uncertainty quantification. Evaluated on a real-world vehicular traffic forecasting task, the proposed STDK GP demonstrates superior performance. Specifically, it achieves a root mean square error of 2.67105 km/h and improves prediction accuracy by approximately 15.28% over the standard GP approach, 21.23% over the state-of-the-art structural recurrent neural network and by more than 53% over stand-alone deep neural networks.

When fine-tuning Deep Neural Networks (DNNs) to new data, DNNs are prone to overwriting network parameters required for task-specific functionality on previously learned tasks, resulting in a loss of performance on those tasks. We propose using parameter-based uncertainty to determine which parameters are relevant to a network’s learned function and regularize training to prevent change in these important parameters. We approach this regularization in two ways: (1), we constrain critical parameters from significant changes by associating more critical parameters with lower learning rates, thereby limiting alterations in those parameters; (2), important parameters are restricted from change by imposing a higher regularization weighting, causing parameters to revert to their states prior to the learning of subsequent tasks. We leverage a Bayesian Moment Propagation framework which learns network parameters concurrently with their associated uncertainties while allowing each parameter to contribute uncertainty to the network’s predictive distribution, avoiding the pitfalls of existing sampling-based methods. The proposed approach is evaluated for common sequential benchmark datasets and compared to existing published approaches from the Continual Learning community. Ultimately, we show improved Continual Learning performance for Average Test Accuracy and Backward Transfer metrics compared to sampling-based methods and other non-uncertainty-based approaches.

Continual learning (CL) focuses on learning non-stationary data distribution without forgetting previous knowledge. Rehearsal-based approaches are commonly used to combat catastrophic forgetting. However, these approaches suffer from a problem called “rehearsal memory overfitting”, where the model becomes too specialized on limited memory samples and loses its ability to generalize effectively. As a result, the effectiveness of the rehearsal memory progressively decays, ultimately resulting in catastrophically forgetting the learned tasks.To address the memory overfitting challenge, we introduce the Adversarially Diversified Rehearsal Memory, or ADRM, a novel method designed to enrich memory sample diversity and bolster resistance against natural and adversarial noise disruptions. ADRM employs the Fast Gradient Sign Method (FGSM) to introduce adversarially modified memory samples, achieving two primary objectives: enhancing memory diversity and fostering a robust response to continual feature drifts in memory samples.We conducted extensive experiments on the CIFAR10 dataset and found that ADRM outperforms several existing CL approaches and performs comparable to state-of-the-art methods. Additionally, we demonstrated ADRM’s ability to enhance CL model robustness under natural and adversarial conditions by using CIFAR10-C and adversarially perturbed CIFAR10 datasets.Our contributions are as follows: Firstly, ADRM addresses overfitting in rehearsal memory by employing FGSM to diversify and increase the complexity of the memory buffer. Secondly, we demonstrate that ADRM mitigates memory overfitting and significantly improves the robustness of CL models, which is crucial for safety-critical applications. Finally, our detailed analysis of features and visualization demonstrates that ADRM mitigates feature drifts in CL memory samples, significantly reducing catastrophic forgetting and resulting in a more resilient CL model. Additionally, our in-depth t-SNE visualizations of feature distribution and the quantification of the feature similarity further enrich our understanding of feature representation in existing CL approaches. Our code is publically available at https://github.com/hikmatkhan/ADRM.

The massive time-series production through the Internet of Things and digital healthcare requires novel data modeling and prediction. Recurrent neural networks (RNNs) are extensively used for analyzing time-series data. However, these models are unable to assess prediction uncertainty, which is particularly critical in heterogeneous and noisy environments. Bayesian inference allows reasoning about predictive uncertainty by estimating the posterior distribution of the parameters. The challenge remains in propagating the high-dimensional distribution through the sequential, non-linear layers of RNNs, resulting in mode collapse leading to erroneous uncertainty estimation and exacerbating the gradient explosion problem. This paper proposes a TRustworthy Uncertainty propagation for Sequential Time-series analysis (TRUST) in RNNs by introducing a Gaussian prior over network parameters and estimating the first two moments of the Gaussian variational distribution using the evidence lower bound. We propagate the variational moments through the sequential, non-linear layers of RNNs using the first-order Taylor approximation. The propagated covariance of the predictive distribution captures uncertainty in the output decision. The extensive experiments using ECG5000 and PeMS-SF classification and weather and power consumption prediction tasks demonstrate 1) significant robustness of TRUST-RNNs against noise and adversarial attacks and 2) self-assessment through the uncertainty that increases significantly with increasing noise.

The superior performance of object detectors is often established under the condition that the test samples are in the same distribution as the training data. However, in many practical applications, out-of-distribution (OOD) instances are inevitable and usually lead to uncertainty in the results. In this paper, we propose a novel, intuitive, and scalable probabilistic object detection method for OOD detection. Unlike other uncertainty-modeling methods that either require huge computational costs to infer the weight distributions or rely on model training through synthetic outlier data, our method is able to distinguish between in-distribution (ID) data and OOD data via weight parameter sampling from proposed Gaussian distributions based on pre-trained networks. We demonstrate that our Bayesian object detector can achieve satisfactory OOD identification performance by reducing the FPR95 score by up to 8.19% and increasing the AUROC score by up to 13.94% when trained on BDD100k and VOC datasets as the ID datasets and evaluated on COCO2017 dataset as the OOD dataset.

Deep neural network (DNN) architectures, such as R-CNN and YOLO, have demonstrated impressive performance in object detection tasks with respect to both time and accuracy. However, detecting small aerial objects remains challenging from both a data and algorithmic perspective. Collecting and annotating videos to detect small aerial objects is a time-consuming task and can quickly become a burden when new classes of objects are added to a database. In addition, the current objective functions for DNNs are not specifically designed for smaller objects. To address these challenges, we propose a virtual reality (VR) dataset for aerial object detection, which can generate large volumes of small-object aerial data. By combining VR data with real-world data, we are able to improve the performance of aerial object detection. We also introduce a cost function derived from the normalized Wasserstein distance to replace the Intersection-over-Union loss for YOLO. Experimental results demonstrate that the VR dataset and normalized Wasserstein distance improve the performance of state-of-the-art object detection methods in detecting small aerial objects. Our source code is publicly available at https://github.com/naddeok96/yolov7_mavrc

Continual learning (CL) is an approach to address catastrophic forgetting, which refers to forgetting previously learned knowledge by neural networks when trained on new tasks or data distributions. The adversarial robustness has decomposed features into robust and non-robust types and demonstrated that models trained on robust features significantly enhance adversarial robustness. However, no study has been conducted on the efficacy of robust features from the lens of the CL model in mitigating catastrophic forgetting in CL. In this paper, we introduce the CL robust dataset and train four baseline models on both the standard and CL robust datasets. Our results demonstrate that the CL models trained on the CL robust dataset experienced less catastrophic forgetting of the previously learned tasks than when trained on the standard dataset. Our observations highlight the significance of the features provided to the underlying CL models, showing that CL robust features can alleviate catastrophic forgetting.

The US lacks a complete national database of private prior permission required airports due to insufficient federal requirements for regular updates. The initial data entry into the system is usually not refreshed by the Federal Aviation Administration (FAA) or local state Department of Transportation. However, outdated or inaccurate information poses risks to aviation safety. This paper suggests a deep learning (DL) approach using Google Earth satellite imagery to identify and locate airport landing sites. The study aims to demonstrate the potential of DL algorithms in processing satellite imagery and improve the precision of the FAA's runway database. We evaluate the performance of Faster Region-based Convolutional Neural Networks using advanced backbone architectures, namely Resnet101 and Resnet-X152, in the detection of airport runways. We incorporate negative samples, i.e., highways images, to enhance the performance of the model. Our simulations reveal that Resnet-X152 outperformed Resnet101 achieving a mean average precision of 76%.

Recent approaches in continual learning (CL) have focused on extracting various types of features from multi-task datasets to prevent catastrophic forgetting - without formally evaluating the quality, robustness and usefulness of these features. Recently, it has been shown that adversarial robustness can be understood by decomposing learned features into robust and non-robust types. The robust features have been used to build robust datasets and have been shown to increase adversarial robustness significantly. There has not been any assessment on using such robust features in CL frameworks to enhance the robustness of CL models against adversarial attacks. Current CL algorithms use standard features - a mixture of robust and non-robust features - and result in models vulnerable to both natural and adversarial noise. This paper presents an empirical study to demonstrate the importance of robust features in the context of class incremental learning (CIL). We adopted the publicly available CIFAR10 dataset for our CIL experiments. We used CIFAR10-Corrupted dataset to evaluate the robustness of the standard, robust and non-robust models against various types of noise including bright-ness, contrast, Gaussian noise and more. To test these models against adversarially attacked input, we created a new dataset using the project gradient descent (PGD) and fast gradient sign (FGSM) algorithm. Our experiments demonstrate that a set of models trained on the standard (a mixture of both robust and non-robust) features obtained a higher accuracy compared to the models trained either using robust features or non-robust features. However, the models trained using standard and non-robust features performed poorly in noisy and adversarial conditions as compared to the model trained using robust features. The model trained using non-robust features performed the worst in noisy conditions and under adversarial attacks. Our study underlines the significance of using robust features in CIL.

Deep learning models have achieved state-of-the-art accuracy in complex tasks, sometimes outperforming human-level accuracy. Yet, they suffer from vulnerabilities known as adversarial attacks, which are imperceptible input perturbations that fool the models on inputs that were originally classified correctly. The adversarial problem remains poorly understood and commonly thought to be an inherent weakness of deep learning models. We argue that understanding and alleviating the adversarial phenomenon may require us to go beyond the Euclidean view and consider the relationship between the input and output spaces as a statistical manifold with the Fisher Information as its Riemannian metric. Under this information geometric view, the optimal attack is constructed as the direction corresponding to the highest eigenvalue of the Fisher Information Matrix - called the Fisher spectral attack. We show that an orthogonal transformation of the data cleverly alters its manifold by keeping the highest eigenvalue but changing the optimal direction of attack; thus deceiving the attacker into adopting the wrong direction. We demonstrate the defensive capabilities of the proposed orthogonal scheme - against the Fisher spectral attack and the popular fast gradient sign method - on standard networks, e.g., LeNet and MobileNetV2 for benchmark data sets, MNIST and CIFAR-10.

Visibility degradation is the cause of many road accidents and air crashes around the globe, it is caused and controlled by multiple key factors that hinder the observer from having a clear vision of what's ahead, playing a crucial role in aviation safety. Alas, researchers and engineers created many tools to measure or restore images suffering from those degradations. This paper compares different image-based deep learning architectures for Atmospheric Visibility Range Classification. A Vision Transformer trained from scratch and three pre-trained Convolution Neural Network (CNN) models were examined and contrasted in terms of accuracy, obtaining a validation accuracy of more than 95 %. Our experiment shows that the different models trained on the Federal Aviation Administration (FAA) initial dataset can be used towards building an efficient tool to aid in Flight Control for long-range visibility estimation. Results showed that DenseNet121 was the best performing model with 99 % training accuracy and 98 % validation accuracy and shows early convergence, while the Vision Transformer kept on showing signs of improving but falling behind only by a bit behind DenseNet121 by the end of the experiment.

The analysis of remote sensing data enables us to detect changes and monitor land surface temperature (LST). However, analysis of times series data poses some challenges, including weather conditions, seasonality and noise, that limit the effectiveness of change detection algorithms. While existing algorithms perform relatively well for detecting abrupt transitions, reliable detection of gradual changes is more difficult. In this paper, we formulate the problem of spatiotemporal LST detection as a statistical sequential change detection problem. LST images are modeled as stochastic processes, with temperature changes reflected as changes in the parameters (i.e., mean) of the process. A generalized likelihood ratio test is used to detect these changes and estimate the exact time/space where they occur. To minimize processing time and memory requirements, we represent LST images by their reduced dimensionality using direct cosine transformation followed by principal component analysis. Statistical sequential analysis is used to provide a unified mathematical framework for the detection of both abrupt and gradual changes in LST observations of Bridgeton Missouri landfill over 17 years.

Progress in speech-related tasks is dependent on the quality of the speech signal being processed. While much progress has been made in various aspects of speech processing (including but not limited to, speech recognition, language detection, and speaker diarization), enhancing a noise-corrupted speech signal as it relates to those tasks has not been rigorously evaluated. Speech enhancement aims to improve the signal-to-noise ratio of a noise-corrupted signal to boost the speech elements (signal) and reduce the non-speech ones (noise). Speech enhancement techniques are evaluated using metrics that are either subjective (asking people their opinion of the enhanced signal) or objective (attempt to calculate metrics based on the signal itself). The subjective measures are better indicators of improved quality but do not scale well to large datasets. The objective metrics have mostly been constructed to attempt to model the subjective results. Our goal in this work is to establish a benchmark to assess the improvement of speech enhancement as it relates to the downstream task of automated speech recognition. In doing so, we retain the qualities of subjective measures while ensuring that evaluation can be done at a large scale in an automated fashion. We explore the impact of various noise types, including stationary, non-stationary, and a shift in noise distribution. We found that existing objective metrics are not a strong indicator of performance as it relates to an improvement in a downstream task. As such, we believe that Word Error Rate should be used when the downstream task is automated speech recognition.

Bayesian neural networks are powerful inference methods by accounting for randomness in the data and the network model. Uncertainty quantification at the output of neural networks is critical, especially for applications such as autonomous driving and hazardous weather forecasting. However, approaches for theoretical analysis of Bayesian neural networks remain limited. This paper makes a step forward towards mathematical quantification of uncertainty in neural network models and proposes a cubature-rule-based computationally-efficient uncertainty quantification approach that captures layer-wise uncertainties of Bayesian neural networks. The proposed approach approximates the first two moments of the posterior distribution of the parameters by propagating cubature points across the network nonlinearities. Simulation results show that the proposed approach can achieve more diverse layer-wise uncertainty quantification results of neural networks with a fast convergence rate.

This paper describes an educational project experience that achieves a software implementation and performance analysis of a blind signal to noise ratio (SNR) estimation system for noisy speech. The system is based on a pattern recognition paradigm and no clean speech reference signal is available. It is a product of the faculty's research and funded by a government contract. The faculty's research on a real-world issue in speech processing has been converted into an undergraduate project. Assessment results show that the project is viewed very favorably by students. Target versus control group results show that the target group feels better qualified for graduate study and career options in digital signal processing.

Gliomas are malignant brain tumors that are associated with high neurological morbidity and poor outcomes. Patients diagnosed with low-grade gliomas are typically followed by a sequence of measurements of the tumor size. Here, we show the promise of Long Short-Term Memory Neural Networks (LSTMs) to address two important clinical questions in low-grade gliomas: 1) classification and prediction of future behavior; and 2) early detection of dedifferentiation to a higher grade or more aggressive growth. We use a system of partial differential equations (PDEs), from our earlier work, to generate simulated growth of low-grade gliomas with different clinical parameters. We design an LSTM network to solve the inverse problem of PDE parameter estimation. We find that accuracy increases as a function of the number of tumor measurements and perplexity can also be used to detect a change in tumor grade. These findings highlight the potential usefulness of LSTMs in solving inverse clinical problems.

Data plenitude is the bottleneck for data-driven approaches, including neural networks. In particular, Convolutional Neural Networks (CNNs) require an abundant database of training images to achieve a desired high accuracy. Current techniques employed for boosting small datasets are data augmentation and synthetic data generation, which suffer from computational complexity and imprecision compared to original datasets. In this paper, we intercalate prior knowledge based on spatial relation between images in the third dimension by computing the gradient of subsequent images in the dataset to remove extraneous information and highlight subtle variations between images. The approach is coined “Inverted Cone” because the volume of brain images below the level of the eyes is ordered to form an inverted cone geometry. The application explored in this work is deboning, or brain extraction, in brain magnetic resonance imaging (MRI) scans. The difficulty of obtaining ground truth for this application prevents the ability of obtaining a large quantity of training images to train the CNN. We considered a limited dataset of 23 patients with and without malignant glioblastoma. Deboning was performed by employing an optimized CNN architecture with and without the Inverted Cone processing. The classic CNN without prior knowledge achieved a validation accuracy of 77%, while the Inverted Cone CNN model achieved a validation accuracy of 86% in a dataset of 451 brain MRI slices.

The power of particle filters in tracking the state of nonlinear and non-Gaussian systems stems not only from their simple numerical implementation but also from their optimality and convergence properties. In particle filtering, the posterior distribution of the state is approximated by a discrete mass of samples, called particles, that stochastically evolve in time according to the dynamics of the model and the observations. Particle filters have been shown to converge almost surely toward the optimal filter as the number of particles increases. However, when additional constraints are imposed, such that every particle must satisfy these constraints, the optimality properties and error bounds of the constrained particle filter remain unexplored. This letter derives performance limits and error bounds of the constrained particle filter. We show that the estimation error is bounded by the area of the state posterior density that does not include the constraining interval. In particular, the error is small if the target density is “well localized” in the constraining interval.

We consider a high-dimension low sample-size multivariate regression problem that accounts for correlation of the response variables. The system is underdetermined as there are more parameters than samples. We show that the maximum likelihood approach with covariance estimation is senseless because the likelihood diverges. We subsequently propose a normalization of the likelihood function that guarantees convergence. We call this method small-sample multivariate regression with covariance (SMURC) estimation. We derive an optimization problem and its convex approximation to compute SMURC. Simulation results show that the proposed algorithm outperforms the regularized likelihood estimator with known covariance matrix and the sparse conditional Gaussian graphical model. We also apply SMURC to the inference of the wing-muscle gene network of the Drosophila melanogaster (fruit fly).

We propose a new non-parametric level set model for automatic image clustering and segmentation based on non-negative matrix factorization (NMF). We show that NMF: (i) clusters the image into distinct homogeneous regions and (ii) provides the local spatial distribution of each region within the image. Furthermore, NMF has a controllable resolution and can discover homogeneous regions as small as one pixel. Coupled with the level-set approach, NMF is an efficient method for image segmentation. The proposed model is unsupervised and relies on local histogram modeling to define an energy functional, whose optimization leads to the final segmentation. A unique and desirable feature of the proposed method is that it does not incorporate any spurious model parameters; hence, the optimization is performed only w.r.t level set functions. We apply the proposed Non-parametrIc Unsupervised SegmentatioN approach (geNIUS) to synthetic and real images and compare it to three state-of-the-art parametric and non-parametric level set approaches: the localized Gaussian distribution fitting model (LGDF) [1], the local histogram fitting (LHF) model [2], and our recent work: NMF-LSM in [3]. The proposed geNIUS model results in a superior accuracy and more efficient implementation, which is a result of its free-model parameter feature.

Electroencephalography (EEG)-based brain computer interface (BCI) is the most studied noninvasive interface to build a direct communication pathway between the brain and an external device. However, correlated noises in EEG measurements still constitute a significant challenge. Alternatively, building BCIs based on filtered brain activity source signals instead of using their surface projections, obtained from the noisy EEG signals, is a promising and not well-explored direction. In this context, finding the locations and waveforms of inner brain sources represents a crucial task for advancing source-based noninvasive BCI technologies. In this paper, we propose a novel multicore beamformer particle filter (multicore BPF) to estimate the EEG brain source spatial locations and their corresponding waveforms. In contrast to conventional (single-core) beamforming spatial filters, the developed multicore BPF considers explicitly temporal correlation among the estimated brain sources by suppressing activation from regions with interfering coherent sources. The hybrid multicore BPF brings together the advantages of both deterministic and Bayesian inverse problem algorithms in order to improve the estimation accuracy. It solves the brain activity localization problem without prior information about approximate areas of source locations. Moreover, the multicore BPF reduces the dimensionality of the problem to half compared with the PF solution, thus alleviating the curse of dimensionality problem. The results, based on generated and real EEG data, show that the proposed framework recovers correctly the dominant sources of brain activity.

This paper presents a new level set method for image segmentation by integrating the level set formulation and the non-negative matrix factorization (NMF). The proposed model characterizes the histogram of the image by dividing the image into blocks and computing the histograms of the blocks as nonnegative combinations of basic histograms. This is achieved by using the NMF algorithm. The basic histograms form a clustering of the image into distinct regions. Our model also takes into account the intensity inhomogeneity or the bias field that usually corrupts medical images. In a level set formulation, this clustering criterion defines an energy in terms of the level set functions that represent a partition of the image domain. The image segmentation is achieved by minimizing this energy with respect to the level set functions and the bias field. Our method is compared, using synthetic and real images, to other state-of-the-art level set approaches that are based on localized clustering and local Gaussian distribution fitting. It is shown that the proposed approach is more robust to noise in the image and intensity inhomogeneity. These advantages stem from the fact that the proposed model i) depends on the distribution of pixels intensities (the histogram) rather than the direct intensity values and ii) does not introduce additional model parameters to be simultaneously estimated with the bias field and the level set functions.

Localization of the brain neural generators that create Electroencephalographs (EEGs) has been an important problem in clinical, research and technological applications related to the brain. The active regions in the brain are modeled as equivalent current dipoles, and the positions and moments of these dipoles or brain sources are estimated. So far, the brain dipoles are assumed to be fixed or time-invariant. However, recent neurological studies are showing that brain sources are not static but vary (in terms of location and moment) depending on various internal and external stimuli. This paper presents a shift in the current paradigm of brain source localization by considering dynamic sources in the brain. We formulate the brain source estimation problem from EEG measurements as a (nonlinear) state-space model. We use the Particle Filter (PF), essentially a sequential Monte Carlo method, to track the trajectory of the moving dipoles in the brain. We further address the “curse of dimensionality,” issue of the PF by taking advantage of the structure of the EEG state-space model, and marginalizing out the linearly evolving states. A Kalman Filter is used to optimally estimate the linear elements, whereas the PF is used to track only the non-linear components. This technique reduces the dimension of the problem; thus exponentially reducing the computational cost. Our simulation results show that, where the PF fails, the Marginalized PF is able to successfully track two dipoles in the brain with only 500 particles.

This paper proposes an algorithm that uses inductive learning and rough set theory (ILRS) to analyze the clinical data available in a patient file (records). A typical patient file has unstructured (both descriptive and quantitative) information that is also uncertain and sometimes incomplete. Successful clinical treatments depend on correct medical diagnosis which determines the correct set of variables or features causing a certain pathology. Clinical applications are by no means the only applications that require decision-making with reasoning from a large and incomplete amount of information. We show that the proposed ILRS technique is able to reduce the available number of features into a smaller core set that precisely describes the information system. We can also quantitatively evaluate the level of dependence of the considered pathology, or decision feature, on a given set of condition features or attributes. Moreover, we show that the proposed algorithm is able to cope with uncertain and incomplete information. We consider a case study of an incomplete information system obtained during cannulation of radial and dorsalis pelis arteries. We show how ILRS succeeds to remove redundancy and determine the most significant condition attributes for a given set of decision attributes from contaminated data with uncertainty. A multi-class classification with preference relations is presented through a set of decision rules. Unlike statistical analysis of clinical data, the reliability of the proposed ILRS algorithm is independent of the data size.

Dynamical systems are often required to satisfy certain constraints arising from basic physical laws, mathematical properties or geometric considerations. Incorporating constraints improves the performance of state estimation and increases the accuracy compared to unconstrained estimation.

In this paper, we propose a statistical approach to reconstruct the brain neuronal activity based only on recorded EEG data. The brain zones with the strongest activity are expressed at a macro level by a few number of active brain dipoles. Normally, for solving the EEG inverse problem, fixed dipole locations are assumed, independently of the different stimuli that excite the brain. The proposed particle filter (PF) framework presents a shift in the current paradigm by estimating dynamic brain dipoles, which may vary from one location to another in the brain depending on internal/external stimuli that may affect the brain. Also, in contrast to previous solutions, the proposed PF algorithm estimates simultaneously, the number of the active dipoles, their moving locations and their respective oscillations in the three dimensional head geometry.

A novel method of classifying data drawn from a nonstationary distribution with drifting mean and variance is presented. The novelty of the approach is based on splitting the problem of tracking a nonstationary distribution into separate classification and time series state estimation problems. State space models for drift in both the mean and variance are presented, which are then successfully tracked using a Kaiman filter and a particle filter for the linear and non-linear parts respectively. Preliminary results, which show the promising potential of the approach, are also presented, along with concluding remarks for potential uses of the proposed approach.

This paper derives a sparse optimal perturbation of gene regulatory networks by determining the optimal perturbation of the minimal number of individual genes that force the network to settle into desired equilibrium states. Previous efforts have led to intervention in gene regulatory networks by deriving the optimal perturbation of the state probability transition matrix. Current technology in molecular biology, however, is limited to perturbation of the state of individual genes, not the state probability transition matrix. Our computer simulation experiments on the Human melanoma gene regulatory network demonstrate the superiority of the proposed approach to gene regulation in comparison to the previous methods based on the marginal of the optimal perturbation of the probability transition matrix of the network.

This is a proof of concept work that proposes a solution to the inverse problem of EEG source estimation by combining two techniques, namely a Particle Filter (PF) for geometrical (3D) localization of the most active brain zones (expressed by two dipoles) and a beamformer (BF) as a spatial filter for estimation of the oscillations that have originated the recorded EEG data. The estimation is reliable for uncorrelated brain sources.

In this paper, we present a comprehensive analysis of self-dual and m-idempotent operators. We refer to an operator as m-idempotent if it converges after m iterations. We focus on an important special case of the general theory of lattice morphology: spatially variant morphology, which captures the geometrical interpretation of spatially variant structuring elements. We demonstrate that every increasing self-dual morphological operator can be viewed as a morphological center. Necessary and sufficient conditions for the idempotence of morphological operators are characterized in terms of their kernel representation. We further extend our results to the representation of the kernel of m-idempotent morphological operators. We then rely on the conditions on the kernel representation derived and establish methods for the construction of m-idempotent and self-dual morphological operators. Finally, we illustrate the importance of the self-duality and m-idempotence properties by an application to speckle noise removal in radar images.

This paper provides an analytical framework to study the performance of linear companding techniques proposed in the OFDM literature, thus settling the numerous controversial claims that are based solely on simulation results. Linear companding transforms are widely employed to reduce the peak-to- average-power ratio (PAPR) in orthogonal frequency division multiplexing (OFDM) systems. Two main linear companding classes have been considered in the literature: linear symmetrical transform (LST) and linear asymmetrical transform (LAST). In the literature, the bit error rate (BER) performance superiority of the basic LAST (with one discontinuity point) over the LST is claimed based on computer simulations. Also, it has been claimed that a LAST with two discontinuity points outperforms the basic LAST with one discontinuity point. These claims are however not substantiated with analytical results. Our analysis shows that these claims are, in general, not always true. We derive a sufficient condition, in terms of the companding parameters, under which the BER performance of a general LAST with M-1 discontinuity points is superior to that of LST. The derived condition explains the contradictions between different reported results in the literature and validates some other reported simulation results. It also serves as a guideline in the process of choosing proper values for companding parameters to obtain a specific trade-off between PAPR reduction capability and BER performance. In particular, the derived sufficient condition shows that the BER performance for LAST depends on the slopes of the LAST rather than on the number of discontinuity points as has been indicated so far. Moreover, we derive conditions for the companding parameters in order to keep the average transmitted power unchanged after companding. Our theoretical derivations are supported by simulation results.

MOTIVATION: Analysis and intervention in the dynamics of gene regulatory networks is at the heart of emerging efforts in the development of modern treatment of numerous ailments including cancer. The ultimate goal is to develop methods to intervene in the function of living organisms in order to drive cells away from a malignant state into a benign form. A serious limitation of much of the previous work in cancer network analysis is the use of external control, which requires intervention at each time step, for an indefinite time interval. This is in sharp contrast to the proposed approach, which relies on the solution of an inverse perturbation problem to introduce a one-time intervention in the structure of regulatory networks. This isolated intervention transforms the steady-state distribution of the dynamic system to the desired steady-state distribution. RESULTS: We formulate the optimal intervention problem in gene regulatory networks as a minimal perturbation of the network in order to force it to converge to a desired steady-state distribution of gene regulation. We cast optimal intervention in gene regulation as a convex optimization problem, thus providing a globally optimal solution which can be efficiently computed using standard toolboxes for convex optimization. The criteria adopted for optimality is chosen to minimize potential adverse effects as a consequence of the intervention strategy. We consider a perturbation that minimizes (i) the overall energy of change between the original and controlled networks and (ii) the time needed to reach the desired steady-state distribution of gene regulation. Furthermore, we show that there is an inherent trade-off between minimizing the energy of the perturbation and the convergence rate to the desired distribution. We apply the proposed control to the human melanoma gene regulatory network. AVAILABILITY: The MATLAB code for optimal intervention in gene regulatory networks can be found online: http://syen.ualr.edu/nxbouaynaya/Bioinformatics2010.html.

In this paper, we propose a communication model of evolution and investigate its information-theoretic bounds. The process of evolution is modeled as the retransmission of information over a protein communication channel, where the transmitted message is the organism's proteome encoded in the DNA. We compute the capacity and the rate distortion functions of the protein communication system for the three domains of life: Archaea, Bacteria, and Eukaryotes. The tradeoff between the transmission rate and the distortion in noisy protein communication channels is analyzed. As expected, comparison between the optimal transmission rate and the channel capacity indicates that the biological fidelity does not reach the Shannon optimal distortion. However, the relationship between the channel capacity and rate distortion achieved for different biological domains provides tremendous insight into the dynamics of the evolutionary processes of the three domains of life. We rely on these results to provide a model of genome sequence evolution based on the two major evolutionary driving forces: mutations and unequal crossovers.

In this paper, we present a general theory of adaptive mathematical morphology (AMM) in the Euclidean space. The proposed theory preserves the notion of a structuring element, which is crucial in the design of geometrical signal and image processing applications. Moreover, we demonstrate the theoretical and practical distinctions between adaptive and spatially-variant mathematical morphology. We provide examples of the use of AMM in various image processing applications, and show the power of the proposed framework in image denoising and segmentation.

Particle filters have revolutionized object tracking in video sequences. The conventional particle filter, also called the CONDENSATION filter, uses the state transition distribution as the proposal distribution, from which the particles are drawn at each iteration. However, the transition distribution does not take into account the current observations, and thus many particles can be wasted in low likelihood regions. One of the most popular methods to improve the performance of particle filters relied on the motion-based proposal density. Although the motivation for motion-based particle filters could be explained on an intuitive level, up until now a mathematical rationale for the improved performance of motion-based particle filters has not been presented. In this letter, we investigate the performance of motion-based particle filters and provide an analytical justification of their superiority over the classical CONDENSATION filter. We rely on the characterization of the optimal proposal density, which minimizes the variance of the particles'weights. However, this density does not admit an analytical expression, making direct sampling from this optimal distribution impossible. We use the Kullback-Leibler (KL) divergence as a similarity measure between density functions and denote a particle filter as superior if the KL divergence between its proposal and the optimal proposal function is lower. We subsequently prove that under mild conditions on the estimated motion vector, the motion-based particle filter outperforms the CONDENSATION filter, in terms of the KL performance measure. Simulation results are presented to support the theoretical analysis.

BACKGROUND: Over the past decade, many investigators have used sophisticated time series tools for the analysis of genomic sequences. Specifically, the correlation of the nucleotide chain has been studied by examining the properties of the power spectrum. The main limitation of the power spectrum is that it is restricted to stationary time series. However, it has been observed over the past decade that genomic sequences exhibit non-stationary statistical behavior. Standard statistical tests have been used to verify that the genomic sequences are indeed not stationary. More recent analysis of genomic data has relied on time-varying power spectral methods to capture the statistical characteristics of genomic sequences. Techniques such as the evolutionary spectrum and evolutionary periodogram have been successful in extracting the time-varying correlation structure. The main difficulty in using time-varying spectral methods is that they are extremely unstable. Large deviations in the correlation structure results from very minor perturbations in the genomic data and experimental procedure. A fundamental new approach is needed in order to provide a stable platform for the non-stationary statistical analysis of genomic sequences. RESULTS: In this paper, we propose to model non-stationary genomic sequences by a time-dependent autoregressive moving average (TD-ARMA) process. The model is based on a classical ARMA process whose coefficients are allowed to vary with time. A series expansion of the time-varying coefficients is used to form a generalized Yule-Walker-type system of equations. A recursive least-squares algorithm is subsequently used to estimate the time-dependent coefficients of the model. The non-stationary parameters estimated are used as a basis for statistical inference and biophysical interpretation of genomic data. In particular, we rely on the TD-ARMA model of genomic sequences to investigate the statistical properties and differentiate between coding and non-coding regions in the nucleotide chain. Specifically, we define a quantitative measure of randomness to assess how far a process deviates from white noise. Our simulation results on various gene sequences show that both the coding and non-coding regions are non-random. However, coding sequences are "whiter" than non-coding sequences as attested by a higher index of randomness. CONCLUSION: We demonstrate that the proposed TD-ARMA model can be used to provide a stable time series tool for the analysis of non-stationary genomic sequences. The estimated time-varying coefficients are used to define an index of randomness, in order to assess the statistical correlations in coding and non-coding DNA sequences. It turns out that the statistical differences between coding and non-coding sequences are more subtle than previously thought using stationary analysis tools: Both coding and non-coding sequences exhibit statistical correlations, with the coding regions being "whiter" than the non-coding regions. These results corroborate the evolutionary periodogram analysis of genomic sequences and revoke the stationary analysis' conclusion that coding DNA behaves like random sequences.

We develop a general theory of spatially-variant (SV) mathematical morphology for binary images in the Euclidean space. The basic SV morphological operators (i.e., SV erosion, SV dilation, SV opening and SV closing) are defined. We demonstrate the ubiquity of SV morphological operators by providing a SV kernel representation of increasing operators. The latter representation is a generalization of Matheron's representation theorem of increasing and translation-invariant operators. The SV kernel representation is redundant, in the sense that a smaller subset of the SV kernel is sufficient for the representation of increasing operators. We provide sufficient conditions for the existence of the minimal basis representation in terms of upper-semi-continuity in the hit-or-miss topology. The latter minimal basis representation is a generalization of Maragos' minimal basis representation for increasing and translation-invariant operators. Moreover, we investigate the upper-semi-continuity property of the basic SV morphological operators. Several examples are used to demonstrate that the theory of spatially-variant mathematical morphology provides a general framework for the unification of various morphological schemes based on spatiallyvariant geometrical structuring elements (e.g., circular, affine and motion morphology). Simulation results illustrate the theory of the proposed spatially-variant morphological framework and show its potential power in various image processing applications.

In this paper, we develop a spatially-variant (SV) mathematical morphology theory for gray-level signals and images in the Euclidean space. The proposed theory preserves the geometrical concept of the structuring function, which provides the foundation of classical morphology and is essential in signal and image processing applications. We define the basic SV gray-level morphological operators (i.e., SV gray-level erosion, dilation, opening, and closing) and investigate their properties. We demonstrate the ubiquity of SV gray-level morphological systems by deriving a kernel representation for a large class of systems, called V-systems, in terms of the basic SV graylevel morphological operators. A V-system is defined to be a gray-level operator, which is invariant under gray-level (vertical) translations. Particular attention is focused on the class of SV flat gray-level operators. The kernel representation for increasing V-systems is a generalization of Maragos' kernel representation for increasing and translation-invariant function-processing systems. A representation of V-systems in terms of their kernel elements is established for increasing and upper-semi-continuous V-systems. This representation unifies a large class of spatially-variant linear and non-linear systems under the same mathematical framework. Finally, simulation results show the potential power of the general theory of gray-level spatially-variant mathematical morphology in several image analysis and computer vision applications.

We derive a new method for multidimensional dynamic programming using the inclusion-exclusion principle. We subsequently propose an extension of the Viterbi algorithm to semi-causal, multidimensional functions. This approach is based on extension of the 1-D trellis structure of the Viterbi algorithm to a tree structure in higher dimensions. We apply the dynamic tree programming algorithm to active surface extraction in video sequences. Simulation results show the efficiency and robustness of the proposed approach

The theory of spatially variant (SV) mathematical morphology is used to extend and analyze two important image processing applications: morphological image restoration and skeleton representation of binary images. For morphological image restoration, we propose the SV alternating sequential filters and SV median filters. We establish the relation of SV median filters to the basic SV morphological operators (i.e., SV erosions and SV dilations). For skeleton representation, we present a general framework for the SV morphological skeleton representation of binary images. We study the properties of the SV morphological skeleton representation and derive conditions for its invertibility. We also develop an algorithm for the implementation of the SV morphological skeleton representation of binary images. The latter algorithm is based on the optimal construction of the SV structuring element mapping designed to minimize the cardinality of the SV morphological skeleton representation. Experimental results show the dramatic improvement in the performance of the SV morphological restoration and SV morphological skeleton representation algorithms in comparison to their translation-invariant counterparts.

The particle filtering framework has revolutionized probabilistic tracking of objects in a video sequence. In this framework, the proposal density can be any density as long as its support includes that of the posterior. However, in practice, the number of samples is finite and consequently the choice of the proposal is crucial to the effectiveness of the tracking. The CONDENSATION filter uses the transition prior as the proposal density. We propose in this paper a motion-based proposal. We use adaptive block matching (ABM) as the motion estimation technique. The benefits of this model are two fold. It increases the sampling efficiency and handles abrupt motion changes. Analytically, we derive a Kullback-Leibler (KL)-based performance measure and show that the motion proposal is superior to the proposal of the CONDENSATION filter. Our experiments are applied to head tracking. Finally, we report promising tracking results in complex environments.

We present a novel automatic system integrating head detection with a particle filter for realtime multi-head tracking (MHT) in video. Distinct from the conventional particle filter, which gets particles from the prior density, we propose a novel importance function based on up to date detection and motion observation which makes the particles more effective and helps us to achieve stable tracking by using much fewer particles. We also propose a general likelihood model in the context of MHT. Different information can be fused in a principled manner to make the tracker more stable. The proposed approach can handle not only changes of scale, lighting, zooming, and pose, but also fast motion, appearance, and hard multi-head occlusion.