We consider the problem of tracking moving targets using mobile wireless sensors (of possibly different types). This is a joint estimation and control problem in which a tracking system must take into account both target and sensor dynamics. We make minimal assumptions about the target dynamics, namely only that their accelerations are bounded. We develop a control law that determines the sensor motion control signals so as to maximize target resolvability as the target dynamics evolve. The method is given a tractable formulation that is amenable to an efficient search method and is evaluated in a series of experiments involving both round-trip time based ranging and Doppler frequency shift measurements. • A unified tracking-control framework for heterogeneous mobile sensors. • The control law improves target resolvability under bounded acceleration. • Sensor control accounts for target uncertainty over a receding horizon. • A tractable formulation enables efficient solution with first-order methods.

We consider the joint problem of online experiment design and parameter estimation for identifying nonlinear system models, while adhering to system constraints. We utilize a receding horizon approach and propose a new adaptive input design criterion, which is tailored to continuously updated parameter estimates, along with a new sequential estimator. We demonstrate the ability of the method to design informative experiments online, while steering the system within operational constraints.

Adversarial training can be used to learn models that are robust against perturbations. For linear models, it can be formulated as a convex optimization problem. Compared to methods proposed in the context of deep learning, leveraging the optimization structure allows significantly faster convergence rates. Still, the use of generic convex solvers can be inefficient for large-scale problems. Here, we propose tailored optimization algorithms for the adversarial training of linear models, which render large-scale regression and classification problems more tractable. For regression problems, we propose a family of solvers based on iterative ridge regression and, for classification, a family of solvers based on projected gradient descent. The methods are based on extended variable reformulations of the original problem. We illustrate their efficiency in numerical examples.

Adversarial training has emerged as a key technique to enhance model robustness against adversarial input perturbations. Many of the existing methods rely on computationally expensive min-max problems that limit their application in practice. We propose a novel formulation of adversarial training in reproducing kernel Hilbert spaces, shifting from input to feature-space perturbations. This reformulation enables the exact solution of inner maximization and efficient optimization. It also provides a regularized estimator that naturally adapts to the noise level and the smoothness of the underlying function. We establish conditions under which the feature-perturbed formulation is a relaxation of the original problem and propose an efficient optimization algorithm based on iterative kernel ridge regression. We provide generalization bounds that help to understand the properties of the method. We also extend the formulation to multiple kernel learning. Empirical evaluation shows good performance in both clean and adversarial settings.

We present an adaptive approach for robust learning from corrupted training sets. We identify corrupted and non-corrupted samples with latent Bernoulli variables and thus formulate the learning problem as maximization of the likelihood where latent variables are marginalized. The resulting problem is solved via variational inference, using an efficient Expectation-Maximization based method. The proposed approach improves over the state-of-the-art by automatically inferring the corruption level, while adding minimal computational overhead. We demonstrate our robust learning method and its parameter-free nature on a wide variety of machine learning tasks including online learning and deep learning where it adapts to different levels of noise and maintains high prediction accuracy.

Randomized trials are widely considered as the gold standard for evaluating the effects of decision policies. Trial data is, however, drawn from a population which may differ from the intended target population and this raises a problem of external validity (aka. generalizability). In this paper we seek to use trial data to draw valid inferences about the outcome of a policy on the target population. Additional covariate data from the target population is used to model the sampling of individuals in the trial study. We develop a method that yields certifiably valid trial-based policy evaluations under any specified range of model miscalibrations. The method is nonparametric and the validity is assured even with finite samples. The certified policy evaluations are illustrated using both simulated and real data.

State-of-the-art machine learning models can be vulnerable to very small input perturbations that are adversarially constructed. Adversarial training is an effective approach to defend against it. Formulated as a min-max problem, it searches for the best solution when the training data were corrupted by the worst-case attacks. Linear models are among the simple models where vulnerabilities can be observed and are the focus of our study. In this case, adversarial training leads to a convex optimization problem which can be formulated as the minimization of a finite sum. We provide a comparative analysis between the solution of adversarial training in linear regression and other regularization methods. Our main findings are that: (A) Adversarial training yields the minimum-norm interpolating solution in the overparameterized regime (more parameters than data), as long as the maximum disturbance radius is smaller than a threshold. And, conversely, the minimum-norm interpolator is the solution to adversarial training with a given radius. (B) Adversarial training can be equivalent to parameter shrinking methods (ridge regression and Lasso). This happens in the underparametrized region, for an appropriate choice of adversarial radius and zero-mean symmetrically distributed covariates. (C) For $\ell_\infty$-adversarial training---as in square-root Lasso---the choice of adversarial radius for optimal bounds does not depend on the additive noise variance. We confirm our theoretical findings with numerical examples.

The paper considers the problem of multi-objective decision support when outcomes are uncertain. We extend the concept of Pareto-efficient decisions to take into account the uncertainty of decision outcomes across varying contexts. This enables quantifying trade-offs between decisions in terms of tail outcomes that are relevant in safety-critical applications. We propose a method for learning efficient decisions with statistical confidence, building on results from the conformal prediction literature. The method adapts to weak or nonexistent context covariate overlap and its statistical guarantees are evaluated using both synthetic and real data.

Conventional methods in causal effect inference typically rely on specifying a valid set of control variables. When this set is unknown or misspecified, inferences will be erroneous. We propose a method for inferring average causal effects when all potential confounders are observed, but the control variables are unknown. When the data-generating process belongs to the class of acyclical linear structural causal models, we prove that the method yields asymptotically valid confidence intervals. Our results build upon a smooth characterization of linear directed acyclic graphs. We verify the capability of the method to produce valid confidence intervals for average causal effects using synthetic data, even when the appropriate specification of control variables is unknown.

We address the problem of learning a decision policy from observational data of past decisions in contexts with features and associated outcomes. The past policy maybe unknown and in safety-critical applications, such as medical decision support, it is of interest to learn robust policies that reduce the risk of outcomes with high costs. In this paper, we develop a method for learning policies that reduce tails of the cost distribution at a specified level and, moreover, provide a statistically valid bound on the cost of each decision. These properties are valid under finite samples -- even in scenarios with uneven or no overlap between features for different decisions in the observed data -- by building on recent results in conformal prediction. The performance and statistical properties of the proposed method are illustrated using both real and synthetic data.

Predictors are learned using past training data which may contain features that are unavailable at the time of prediction. We develop an approach that is robust against outlying missing features, based on the optimality properties of an oracle predictor which observes them. The robustness properties of the approach are demonstrated on both real and synthetic data.

We address the problem of inferring the causal effect of an exposure on an outcome across space, using observational data. The data is possibly subject to unmeasured confounding variables which, in a standard approach, must be adjusted for by estimating a nuisance function. Here we develop a method that eliminates the nuisance function, while mitigating the resulting errors-in-variables. The result is a robust and accurate inference method for spatially varying heterogeneous causal effects. The properties of the method are demonstrated on synthetic as well as real data from Germany and the US.

A spatial point process can be characterized by an intensity function which predicts the number of events that occur across space. In this paper, we develop a method to infer predictive intensity intervals by learning a spatial model using a regularized criterion. We prove that the proposed method exhibits out-of-sample prediction performance guarantees which, unlike standard estimators, are valid even when the spatial model is misspecified. The method is demonstrated using synthetic as well as real spatial data.

In safety-critical applications a probabilistic model is usually required to be calibrated, i.e., to capture the uncertainty of its predictions accurately. In multi-class classification, calibration of the most confident predictions only is often not sufficient. We propose and study calibration measures for multi-class classification that generalize existing measures such as the expected calibration error, the maximum calibration error, and the maximum mean calibration error. We propose and evaluate empirically different consistent and unbiased estimators for a specific class of measures based on matrix-valued kernels. Importantly, these estimators can be interpreted as test statistics associated with well-defined bounds and approximations of the p-value under the null hypothesis that the model is calibrated, significantly improving the interpretability of calibration measures, which otherwise lack any meaningful unit or scale.

We address the problem of prediction of multivariate data process using an underlying graph model. We develop a method that learns a sparse partial correlation graph in a tuning-free and computationally efficient manner. Specifically, the graph structure is learned recursively without the need for cross validation or parameter tuning by building upon a hyperparameter-free framework. Our approach does not require the graph to be undirected and also accommodates varying noise levels across different nodes. Experiments using real-world datasets show that the proposed method offers significant performance gains in prediction, in comparison with the graphs frequently associated with these datasets.

In many applications, different populations are compared using data that are sampled in a biased manner. Under sampling biases, standard methods that estimate the difference between the population means yield unreliable inferences. Here, we develop an inference method that is resilient to sampling biases and is able to control the false positive errors under moderate bias levels in contrast to the standard approach. We demonstrate the method using synthetic and real biomarker data.

We address the problem of predicting spatio-temporal processes with temporal patterns that vary across spatial regions, when data is obtained as a stream. That is, when the training dataset is augmented sequentially. Specifically, we develop a localized spatio-temporal covariance model of the process that can capture spatially varying temporal periodicities in the data. We then apply a covariance-fitting methodology to learn the model parameters which yields a predictor that can be updated sequentially with each new data point. The proposed method is evaluated using both synthetic and real climate data which demonstrate its ability to accurately predict data missing in spatial regions over time.

This paper considers the quantification of the prediction performance in Gaussian process regression. The standard approach is to base the prediction error bars on the theoretical predictive variance, which is a lower bound on the mean square-error (MSE). This approach, however, does not take into account that the statistical model is learned from the data. We show that this omission leads to a systematic underestimation of the prediction errors. Starting from a generalization of the Cramér-Rao bound, we derive a more accurate MSE bound which provides a measure of uncertainty for prediction of Gaussian processes. The improved bound is easily computed and we illustrate it using synthetic and real data examples.

Clock synchronization is ubiquitous in wireless systems for communication, sensing, and control. In this paper, we design a scalable system in which an indefinite number of passively receiving wireless units can synchronize to a single master clock at the level of discrete clock ticks. Accurate synchronization requires an estimate of the node positions to compensate the time-of-flight transmission delay in line-of-sight environments. If such information is available, the framework developed here takes position uncertainties into account. In the absence of such information, as in indoor scenarios, we propose an auxiliary localization mechanism. Furthermore, we derive the Cramer-Rao bounds for the system, which show that it enables synchronization accuracy at sub-nanosecond levels. Finally, we develop and evaluate an online estimation method, which is statistically efficient.

In this paper, we develop a new technique for estimating fine clock errors and range between two nodes simultaneously by two-way time-of-arrival measurements using impulse-radio ultrawideband signals. Estimators for clock parameters and the range are proposed, which are robust with respect to outliers. They are analyzed numerically and by means of experimental measurement campaigns. The technique and derived estimators achieve accuracies below 1 Hz for frequency estimation, below 1 ns for phase estimation, and 20 cm for range estimation, at a 4-m distance using 100-MHz clocks at both nodes. Therefore, we show that the proposed joint approach is practical and can simultaneously provide clock synchronization and positioning in an experimental system.

In this paper, we propose a robust transmit beampattern design for multiple-input multiple-output (MIMO) radar systems. The objective considered here is minimization of the beampattern sidelobes, subject to constraints on the transmit power where the waveform co-variance matrix is the optimization variable. Motivated by the fact that the steering vectors are subject to uncertainties in practice, we propose a worst-case robust beampattern design where the uncertainties are parameterized by a deterministic set. We show that the resulting non-convex maximin problem can be translated into a convex problem. We numerically illustrate that the steering vector uncertainty yields a severe degradation in the array performance, i.e., the transmit beampattern. Also, we show that the proposed robust design improves the transmit beampattern by reducing the worst case sidelobe peak levels.

For compressive sensing, we derive achievable performance guarantees for recovering partial support sets of sparse vectors. The guarantees are determined in terms of the fraction of signal power to be detected and the measurement rate, defined as a relation between the dimensions of the measurement matrix. Based on this result we derive a tradeoff between the measurement rate and the mean square error, and illustrate it by a numerical example.

In a compressed sensing setup with jointly sparse, correlated data, we develop a distributed greedy algorithm called distributed predictive subspace pursuit. Based on estimates from neighboring sensor nodes, this algorithm operates iteratively in two steps: first forming a prediction of the signal and then solving the compressed sensing problem with an iterative linear minimum mean squared estimator. Through simulations we show that the algorithm provides better performance than current state-of-the-art algorithms.

This paper considers the problem of reconstructing low-rank matrices from undersampled measurements, when the matrix has a known linear structure. Based on the iterative reweighted least-squares approach, we develop an algorithm that exploits the linear structure in an efficient way that allows for reconstruction in highly undersampled scenarios. The method also enables inferring an appropriate regularization parameter value from the observations. The performance of the method is tested in a missing data recovery problem.

We investigate a wireless network localization scenario in which the need for synchronized nodes is avoided. It consists of a set of fixed anchor nodes transmitting according to a given sequence and a self-localizing receiver node. The setup can accommodate additional nodes with unknown positions participating in the sequence. We propose a localization method which is robust with respect to uncertainty of the anchor positions and other system parameters. Further, we investigate the Cramér-Rao bound for the considered problem and show through numerical simulations that the proposed method attains the bound.

For array processing, we consider the problem of estimating signals of interest, and their directions of arrival (DOA), in unknown colored noise fields. We develop an estimator that efficiently utilizes a set of noise-only samples and, further, can incorporate prior knowledge of the DOAs with varying degrees of certainty. The estimator is compared with state of the art estimators that utilize noise-only samples, and the Cramér-Rao bound, exhibiting improved performance for smaller sample sets and in poor signal conditions.

For compressive sensing of dynamic sparse signals, we develop an iterative greedy search algorithm based on subspace pursuit (SP) that can incorporate sequential predictions, thereby taking advantage of its low complexity while improving recovery performance by exploiting correlations described by a state space model. The algorithm, which we call dynamic subspace pursuit (DSP), is presented and experimentally validated. It exhibits a graceful degradation at deteriorating signal conditions while capable of yielding substantial performance gains as conditions improve.

A method is proposed to fuse the information from two navigation systems whose relative position is unknown, but where there exists an upper limit on how far apart the two systems can be. The proposed information fusion method is applied to a scenario in which a pedestrian is equipped with two foot-mounted zero-velocity-aided inertial navigation systems; one system on each foot. The performance of the method is studied using experimental data. The results show that the method has the capability to significantly improve the navigation performance when compared to using two uncoupled foot-mounted systems.

Pedestrian Dead-Reckoning (PDR) and Radio Frequency (RF) ranging/positioning are complementary techniques for position estimation but they usually locate different points in the body (RF in the head/hand and PDR in the foot). We propose to fuse the information from both navigation points using a constraint filter with an upper bound in the distance between the estimated positions of both sensors.

For reconstruction of low-rank matrices from undersampled measurements, we develop an iterative algorithm based on least-squares estimation. While the algorithm can be used for any low-rank matrix, it is also capable of exploiting a-priori knowledge of matrix structure. In particular, we consider linearly structured matrices, such as Hankel and Toeplitz, as well as positive semidefinite matrices. The performance of the algorithm, referred to as alternating least-squares (ALS), is evaluated by simulations and compared to the Cramér-Rao bounds.

We consider the problem of estimating a random state vector when there is information about the maximum distances between its subvectors. The estimation problem is posed in a Bayesian framework in which the minimum mean square error (MMSE) estimate of the state is given by the conditional mean. Since finding the conditional mean requires multidimensional integration, an approximate MMSE estimator is proposed. The performance of the proposed estimator is evaluated in a positioning problem. Finally, the application of the estimator in inequality constrained recursive filtering is illustrated by applying the estimator to a dead-reckoning problem. The MSE of the estimator is compared with two related posterior Cramér-Rao bounds.

For small-scale Unmanned Aerial Vehicles (UAV) to operate indoor, in urban canyons or other scenarios where signals from global navigation satellite systems are denied or impaired, alternative estimation and control strategies must be applied. In this paper a system is proposed that estimates the self-motion and wind velocity by fusing information from airspeed sensors, an inertial measurement unit (IMU) and a monocular camera. Such estimates can be used in control systems for managing wind disturbances or chemical plume based tracking strategies. Simulation results indicate that while the inertial dead-reckoning process is subject to drift, the system is capable of separating the self-motion and wind velocity from the airspeed information.

An estimation procedure for calibration of a low-cost inertial measurement unit (IMU), using a rigidly mounted monocular camera, is presented. The parameters of a sensor model that captures misalignments, scale and offset errors are estimated jointly with the IMU-camera coordinate transformation parameters using a recursive Sigma-Point Kalman Filter. The method requires only a simple visual calibration pattern. A simulation study indicates the filter's ability to reach subcentimeter and subdegree accuracy.

We introduce a method to improve performance of multiple-description coding based on legacy video coders with pre-and postprocessing. The pre-and post-processing setup is general, making the method applicable to most legacy coders. For the case of two coders, a relative displacement of the intra-coding mode between the coders is shown to give improved robustness to packet loss. The optimal displacement of the intra-coding mode is found analytically, using a distortion minimization formulation where two independent Gilbert channels are assumed. The analytical results are confirmed by simulations. Tests with an H.263 coder show significant improvement in YPSNR over equivalent systems with no relative displacement of the intra-coding operation.