Signal processing makes extensive use of point estimators and accompanying error bounds. These work well up until the likelihood function has two or more high peaks. When it is important for an estimator to remain reliable, it becomes necessary to consider alternatives, such as set estimators. An obvious first choice might be confidence intervals or confidence regions, but there can be difficulties in computing and interpreting them (and sometimes they might still be blind to multiple peaks in the likelihood). Bayesians seize on this to argue for replacing confidence regions with credible regions. Yet Bayesian statistics require a prior, which is not always a natural part of the problem formulation. This paper demonstrates how a reinterpretation of the prior as a weighting function makes an otherwise Bayesian estimator meaningful in the frequentist context. The weighting function interpretation also serves as a reminder that an estimator should always be designed in the context of its intended application; unlike a prior which ostensibly depends on prior knowledge, a weighting function depends on the intended application. This paper uses the time-of-arrival (TOA) problem to illustrate all these points. It also derives a basic theory of region-based estimators distinct from confidence regions.

The detection of irregularly spaced pulses of non-negligible width is a fascinating yet under-explored topic in signal processing. It sits adjacent to other core topics such as radar and symbol detection yet has its own distinctive challenges. Even modern techniques such as compressed sensing perform worse than may be expected on pulse processing problems. Real-world applications include nuclear spectroscopy, flow cytometry, seismic signal processing and neural spike sorting, and these in turn have applications to environmental radiation monitoring, surveying, diagnostic medicine, industrial imaging, biomedical imaging, top-down proteomics, and security screening, to name just a few. This overview paper endeavours to position the pulse processing problem in the context of signal processing. It also describes some current challenges in the field.

The generalization error of a learning algorithm refers to the discrepancy between the loss of a learning algorithm on training data and that on unseen testing data. Various information-theoretic bounds on the generalization error have been derived in the literature, where the mutual information between the training data and the hypothesis (the output of the learning algorithm) plays an important role. Focusing on the individual sample mutual information bound by Bu et al. [2], which itself is a tightened version of the first bound on the topic by Russo et al. [3] and Xu et al. [4], this paper investigates the tightness of these bounds, in terms of the dependence of their convergence rates on the sample size n. It has been recognized that these bounds are in general not tight, readily verified for the exemplary quadratic Gaussian mean estimation problem, where the individual sample mutual information bound scales asO(√1/n) while the true generalization error scales asO(1/n). The first contribution of this paper is to show that the same bound can in fact be asymptotically tight if an appropriate assumption is made. In particular, we show that the fast rate can be recovered when the assumption is made on the excess risk instead of the loss function, which was usually done in existing literature. A theoretical justification is given for this choice. The second contribution of the paper is a new set of generalization error bounds based on the (η,c)-central condition, a condition relatively easy to verify and has the property that the mutual information term directly determines the convergence rate of the bound. Several analytical and numerical examples are given to show the effectiveness of these bounds.

This study introduces an angle-based micro-Doppler analysis using Frequency Modulated Continuous Wave (FMCW) radar tailored for axle-based vehicle classification. The novel approach exploits the signal angle of arrival to separate incoming signals and noise from distinct targets. This is done by analysing the phase difference of a dual antenna radar system based on the time-frequency representation of the radar beat signal. Vehicles driving side by side can now be discriminated. Multipath signals and clutter are more easily identified and filtered out. This paper extends the use of radar systems for non-invasive axle-based vehicle classification by improving the estimation of vehicle features. The method’s effectiveness is demonstrated using real traffic data on vehicle classification performance.

Recent advancements in unsupervised domain adaptation (UDA) and semi-supervised learning (SSL), particularly incorporating causality, have led to significant methodological improvements in these learning problems. However, a formal theory that explains the role of causality in the generalization performance of UDA/SSL is still lacking. In this paper, we consider the UDA/SSL scenarios where we access $m$ labelled source data and $n$ unlabelled target data as training instances under different causal settings with a parametric probabilistic model. We study the learning performance (e.g., excess risk) of prediction in the target domain from an information-theoretic perspective. Specifically, we distinguish two scenarios: the learning problem is called causal learning if the feature is the cause and the label is the effect, and is called anti-causal learning otherwise. We show that in causal learning, the excess risk depends on the size of the source sample at a rate of $O(\frac{1}{m})$ only if the labelling distribution between the source and target domains remains unchanged. In anti-causal learning, we show that the unlabelled data dominate the performance at a rate of typically $O(\frac{1}{n})$. These results bring out the relationship between the data sample size and the hardness of the learning problem with different causal mechanisms.

Transfer learning, or domain adaptation, is concerned with machine learning problems in which training and testing data come from possibly different probability distributions. In this work, we give an information-theoretic analysis of the generalization error and excess risk of transfer learning algorithms. Our results suggest, perhaps as expected, that the Kullback-Leibler (KL) divergence$D(\mu \|\mu ')$plays an important role in the characterizations where$\mu $and$\mu '$denote the distribution of the training data and the testing data, respectively. Specifically, we provide generalization error and excess risk upper bounds for learning algorithms where data from both distributions are available in the training phase. Recognizing that the bounds could be sub-optimal in general, we provide improved excess risk upper bounds for a certain class of algorithms, including the empirical risk minimization (ERM) algorithm, by making stronger assumptions through the central condition. To demonstrate the usefulness of the bounds, we further extend the analysis to the Gibbs algorithm and the noisy stochastic gradient descent method. We then generalize the mutual information bound with other divergences such as$\phi $-divergence and Wasserstein distance, which may lead to tighter bounds and can handle the case when$\mu $is not absolutely continuous with respect to$\mu '$. Several numerical results are provided to demonstrate our theoretical findings. Lastly, to address the problem that the bounds are often not directly applicable in practice due to the absence of the distributional knowledge of the data, we develop an algorithm (called InfoBoost) that dynamically adjusts the importance weights for both source and target data based on certain information measures. The empirical results show the effectiveness of the proposed algorithm.

Transfer learning is a machine learning paradigm where knowledge from one problem is utilized to solve a new but related problem. While conceivable that knowledge from one task could help solve a related task, if not executed properly, transfer learning algorithms can impair the learning performance instead of improving it – commonly known as negative transfer. In this paper, we use a parametric statistical model to study transfer learning from a Bayesian perspective. Specifically, we study three variants of transfer learning problems, instantaneous, online, and time-variant transfer learning. We define an appropriate objective function for each problem and provide either exact expressions or upper bounds on the learning performance using information-theoretic quantities, which allow simple and explicit characterizations when the sample size becomes large. Furthermore, examples show that the derived bounds are accurate even for small sample sizes. The obtained bounds give valuable insights into the effect of prior knowledge on transfer learning, at least with respect to our Bayesian formulation of the transfer learning problem. In particular, we formally characterize the conditions under which negative transfer occurs. Lastly, we devise several (online) transfer learning algorithms that are amenable to practical implementations, some of which do not require the parametric assumption. We demonstrate the effectiveness of our algorithms with real data sets, focusing primarily on when the source and target data have strong similarities.

Approximate Message Passing (AMP) is an efficient iterative parameter-estimation technique for certain high-dimensional linear systems with non-Gaussian distributions, such as sparse systems. In AMP, a so-called Onsager term is added to keep estimation errors approximately Gaussian. Orthogonal AMP (OAMP) does not require this Onsager term, relying instead on an orthogonalization procedure to keep the current errors uncorrelated with (i.e., orthogonal to) past errors. In this paper, we show the generality and significance of the orthogonality in ensuring that errors are “asymptotically independently and identically distributed Gaussian” (AIIDG). This AIIDG property, which is essential for the attractive performance of OAMP, holds for separable functions. We present a simple and versatile procedure to establish the orthogonality through Gram-Schmidt (GS) orthogonalization, which is applicable to any prototype. We show that different AMP-type algorithms, such as expectation propagation (EP), turbo, AMP and OAMP, can be unified under the orthogonal principle. The simplicity and generality of OAMP provide efficient solutions for estimation problems beyond the classical linear models. As an example, we study the optimization of OAMP via the GS model and GS orthogonalization. More related applications will be discussed in a companion paper where new algorithms are developed for problems with multiple constraints and multiple measurement variables.

A recent line of works, initiated by [1] and [2], has shown that the generalization error of a learning algorithm can be upper bounded by information measures. In most of the relevant works, the convergence rate of the expected generalization error is in the form of $O(\sqrt {\lambda I/n} )$ where λ is an assumption-dependent coefficient and I is some information-theoretic quantities such as the mutual information between the data sample and the learned hypothesis. However, such a learning rate is typically considered to be "slow", compared to a "fast rate" of O(1 /n) in many learning scenarios. In this work, we first show that the square root does not necessarily imply a slow rate, and a fast rate result can still be obtained using this bound by evaluating λ under an appropriate assumption. Furthermore, we identify the key conditions needed for the fast rate generalization error, which we call the ( η, c)-central condition. Under this condition, we give information-theoretic bounds on the generalization error and excess risk, with a convergence rate of O (1 /n) for specific learning algorithms such as empirical risk minimization. Finally, analytical examples are given to show the effectiveness of the bounds.

Geometric deep learning is a relatively nascent field that has attracted significant attention in the past few years. This is partly due to the availability of data acquired from non-euclidean domains or features extracted from euclidean-space data that reside on smooth manifolds. For instance, pose data commonly encountered in computer vision reside in Lie groups, while covariance matrices that are ubiquitous in many fields and diffusion tensors encountered in medical imaging domain reside on the manifold of symmetric positive definite matrices. Much of this data is naturally represented as a grid of manifold-valued data. In this paper we present a novel theoretical framework for developing deep neural networks to cope with these grids of manifold-valued data inputs. We also present a novel architecture to realize this theory and call it the ManifoldNet. Analogous to vector spaces where convolutions are equivalent to computing weighted sums, manifold-valued data 'convolutions' can be defined using the weighted Fréchet Mean ([Formula: see text]). (This requires endowing the manifold with a Riemannian structure if it did not already come with one.) The hidden layers of ManifoldNet compute [Formula: see text]s of their inputs, where the weights are to be learnt. This means the data remain manifold-valued as they propagate through the hidden layers. To reduce computational complexity, we present a provably convergent recursive algorithm for computing the [Formula: see text]. Further, we prove that on non-constant sectional curvature manifolds, each [Formula: see text] layer is a contraction mapping and provide constructive evidence for its non-collapsibility when stacked in layers. This captures the two fundamental properties of deep network layers. Analogous to the equivariance of convolution in euclidean space to translations, we prove that the [Formula: see text] is equivariant to the action of the group of isometries admitted by the Riemannian manifold on which the data reside. To showcase the performance of ManifoldNet, we present several experiments using both computer vision and medical imaging data sets.

Transfer learning is a machine learning paradigm where the knowledge from one task is utilized to resolve the problem in a related task. On the one hand, it is conceivable that knowledge from one task could be useful for solving a related problem. On the other hand, it is also recognized that if not executed properly, transfer learning algorithms could in fact impair the learning performance instead of improving it - commonly known as negative transfer. In this paper, we study the online transfer learning problems where the source samples are given in an off-line way while the target samples arrive sequentially. We define the expected regret of the online transfer learning problem, and provide upper bounds on the regret using information-theoretic quantities. We also obtain exact expressions for the bounds when the sample size becomes large. Examples show that the derived bounds are accurate even for small sample sizes. Furthermore, the obtained bounds give valuable insight on the effect of prior knowledge for transfer learning in our formulation. In particular, we formally characterize the conditions under which negative transfer occurs.

Hopfield neural networks are a possible basis for modelling associative memory in living organisms. After summarising previous studies in the field, we take a new look at learning rules, exhibiting them as descent-type algorithms for various cost functions. We also propose several new cost functions suitable for learning. We discuss the role of biases - the external inputs - in the learning process in Hopfield networks. Furthermore, we apply Newton's method for learning memories, and experimentally compare the performances of various learning rules. Finally, to add to the debate whether allowing connections of a neuron to itself enhances memory capacity, we numerically investigate the effects of self-coupling.

Transfer learning, or domain adaptation, is concerned with machine learning problems in which training and testing data come from possibly different distributions (denoted as μ and μ', respectively). In this work, we give an informationtheoretic analysis on the generalization error and the excess risk of transfer learning algorithms, following a line of work initiated by Russo and Zhou. Our results suggest, perhaps as expected, that the Kullback-Leibler (KL) divergence D(μ||μ') plays an important role in characterizing the generalization error in the settings of domain adaptation. Specifically, we provide generalization error upper bounds for general transfer learning algorithms, and extend the results to a specific empirical risk minimization (ERM) algorithm where data from both distributions are available in the training phase. We further apply the method to iterative, noisy gradient descent algorithms, and obtain upper bounds which can be easily calculated, only using parameters from the learning algorithms. A few illustrative examples are provided to demonstrate the usefulness of the results. In particular, our bound is tighter in specific classification problems than the bound derived using Rademacher complexity.

We investigate the existence question for maximum-likelihood estimation for a time-of-arrival-based localization problem where the signal transmission time is unknown. We have previously shown that for some inputs, the maximum-likelihood estimator (MLE) does not exist. Here we go further: under a genericity condition, we prove that there is nonzero probability that the MLE does not exist. Besides this theoretical result, we draw attention to practical issues with finding the estimator using local optimization techniques.

A class of discrete-time random processes that have seen a wide variety of applications consists of a linear state-space model whose parameters are modulated by the state of a finite-state Markov chain. A typical way to filter such processes is with collapsing methods, which approximate the underlying distribution by a mixture of Gaussians indexed by the recent history of the Markov chain. The computational cost of such methods increases rapidly as the error decreases to zero. This paper presents an alternative approach to filtering these processes based on keeping track of the values of the underlying probability density function and characteristic function on grids. It has favourable convergence properties under certain assumptions.

International audience

Data, which lie in the space Pm, of m × m symmetric positive definite matrices, (sometimes called tensor data), play a fundamental role in applications, including medical imaging, computer vision, and radar signal processing. An open challenge, for these applications, is to find a class of probability distributions, which is able to capture the statistical properties of data in Pm, as they arise in real-world situations. The present paper meets this challenge by introducing Riemannian Gaussian distributions on Pm. Distributions of this kind were first considered by Pennec in 2006. However, the present paper gives an exact expression of their probability density function for the first time in existing literature. This leads to two original contributions. First, a detailed study of statistical inference for Riemannian Gaussian distributions, uncovering the connection between the maximum likelihood estimation and the concept of Riemannian centre of mass, widely used in applications. Second, the derivation and the implementation of an expectation-maximisation algorithm, for the estimation of mixtures of Riemannian Gaussian distributions. The paper applies this new algorithm, to the classification of data in Pm, (concretely, to the problem of texture classification, in computer vision), showing that it yields significantly better performance, in comparison to recent approaches.

Single-particle imaging experiments using X-ray Free-Electron Lasers (XFEL) belong to a new generation of X-ray imaging techniques potentially allowing high resolution images of non-crystallizable molecules to be obtained. One of the challenges of single-particle imaging is the reconstruction of the 3D intensity function from only a few samples collected on a planar detector after the interaction of a free falling molecule and the X-ray beam. In this paper, we take advantage of the symmetries of the intensity function to propose an original low-resolution reconstruction algorithm based on an Expansion Maximization Compression (EMC) approach. We study the problem of adequate sampling of the rotation group via simulation to illustrate the potential of the approach.

This paper presents several results about isotropic random walks and multiple scattering processes on hyperspheres Sp-1. It allows one to derive the Fourier expansions on Sp-1of these processes. A result of unimodality for the multiconvolution of symmetrical probability density functions on Sp-1is also introduced. Such processes are then studied in the case where the scattering distribution is von Mises-Fisher (vMF). Asymptotic distributions for the multiconvolution of vMFs on Sp-1are obtained. Both Fourier expansion and asymptotic approximation allow us to compute estimation bounds for the parameters of compound cox processes on Sp-1.

Whether it be a road, railway track, pipeline, or fiber-optic cable, achieving a low probability of “cable” damage is important for the proper operation of modern infrastructure. This paper considers the fundamental problem of how to connect two points with a cable that crosses an earthquake fault line. We first develop a model, under certain general assumptions, for the cable break probability. Then, we formulate a multiobjective optimization problem, with cable cost and probability of cable break being the two objectives. For two important and meaningful sets of cable shape alternatives, the Pareto front is determined for these two objectives. All the analytical results are verified by simulations.

This paper considers the problem of parameter estimation for multiple scattering process on the sphere. Using harmonic analysis, a Fourier expansion of the pdf of the process is obtained. Based on the Fourier coefficient statistics, we consider the problem of estimating the parameter of the process using an Approximate Bayesian Computation (ABC) approach. Simulations show the ability of the proposed approach for the density estimation of intensity and concentration parameters for the von Mises Fisher multiple scattering process.

This paper describes the application of particle filtering to the solution of the problem of filtering with observations in a manifold. Mathematically, this is based on an original use of so-called connector maps. It is shown that well-chosen connector maps can be used to transform successive samples from a continuous time observation process, evolving on a manifold, into a discrete sequence of random vectors, which are asymptotically independent and normally distributed, in the limit where the sampling interval goes to zero. Roughly speaking, this “innovation sequence” can be used as the input of a sequential Monte Carlo algorithm. As a concrete application, numerical simulation results are presented, for the problem of estimating the angular velocity of a rigid body from noisy observations of its attitude.

Partial observation of stochastic processes can occur for various reasons, ranging from faulty sensors to occultation issues. In this paper, we consider the problem of estimating the angular velocity of a rotating system from partial observation corrupted by noise. The system is assumed to evolve on the rotation group SO(n), and only k noisy measurements with k <; n are available. We propose an optimal filter to track the angular velocity. We show that, under some conditions, it is possible to recover the angular velocity of the rotating system and we propose a solution based on a Monte-Carlo method (particle filter). In particular, we show that if the angular velocity is stepwise constant, our algorithm succeed in estimating it. Simulations illustrate the proposed approach.

Bayesian spiking neurons (BSNs) provide a probabilistic interpretation of how neurons perform inference and learning. Online learning in BSNs typically involves parameter estimation based on maximum-likelihood expectation-maximization (ML-EM) which is computationally slow and limits the potential of studying networks of BSNs. An online learning algorithm, fast learning (FL), is presented that is more computationally efficient than the benchmark ML-EM for a fixed number of time steps as the number of inputs to a BSN increases (e.g., 16.5 times faster run times for 20 inputs). Although ML-EM appears to converge 2.0 to 3.6 times faster than FL, the computational cost of ML-EM means that ML-EM takes longer to simulate to convergence than FL. FL also provides reasonable convergence performance that is robust to initialization of parameter estimates that are far from the true parameter values. However, parameter estimation depends on the range of true parameter values. Nevertheless, for a physiologically meaningful range of parameter values, FL gives very good average estimation accuracy, despite its approximate nature. The FL algorithm therefore provides an efficient tool, complementary to ML-EM, for exploring BSN networks in more detail in order to better understand their biological relevance. Moreover, the simplicity of the FL algorithm means it can be easily implemented in neuromorphic VLSI such that one can take advantage of the energy-efficient spike coding of BSNs.

A block-wise constellation design is presented for optical communication systems with multi-subcarrier modulation (MSM), intensity modulation (IM) and direct detection (DD). The DC-bias traditionally used only for compensating the negative peaks of the transmitter-side signals is treated as an information-carrying basis in our proposed scheme called MSM-JDCM. Designs are done for both flat-fading and frequency selective-fading scenarios, and following a principle of high dimensional sphere packing. To simplify the problem, we apply the following methods. First, we use bounds on the waveform's maximum and minimum. Second, we use the maximum and minimum constraints on a set of sufficient samples of waveforms. Third, we relax non-convex distance constraints into convex ones by iterative linearizations. With the MSM-JDCM, we minimize electrical power, optical power, and peak power with a common target bit error rate (BER). Analysis shows that the MSM-JDCM offers significant power gains over MSM-Normal and MSM-SPSS. The short-term peak to average power ratio (PAPR) and long-term PAPR constraints are combined with the MSM-JDCM to mitigate the nonlinear distortion caused by high power amplifier and laser diode, which is another novelty of our scheme. To attain lower BER, a binary switching algorithm (BSA) is applied to find the improved constellation labeling.

Technological improvements have led to situations where data sets are sufficiently rich that in the interests of processing speed it is desirable to throw away samples that provide little additional information. This is referred to here as data sparsification. The first contribution is a study of a recently proposed data sparsification scheme; ideas from vector quantisation are used to assess its performance. Informed by this study, a modification of the data sparsification algorithm is proposed and applied to the problem of estimating a kernel-based measure of independence of two datasets. (Given i.i.d. observations from two random variables, x and y, the underlying problem is to determine whether or not x and y are independent of each other.) The second contribution of this paper is to make recursive an existing algorithm for measuring independence and able to operate on both raw data and on sparsified data generated by the aforementioned data sparsification algorithm. Compared with the original algorithm, the recursive algorithm is significantly faster due to its lower memory and computational requirements.

Wide sense stationary processes are a mainstay of classical signal processing. It is well known that they can be obtained by solving ordinary differential equations with constant coefficients whose right-hand side is a white noise. This paper addresses the extension of this construction to random fields defined on compact Lie groups. On an underlying compact Lie group, the paper studies left invariant second-order elliptic partial differential equations whose right-hand side is a spatial white noise. Quite often, the solution of a partial differential equation is not defined as a function but as a distribution. To adapt to this situation, the paper introduces a definition of wide sense stationary distributions on a compact Lie group. This is shown to be consistent with the more restricted definition of wide sense stationary fields given in a classic paper by Yaglom. It is proved that the solution of a partial differential equation, of the kind being studied, is a wide sense stationary distribution whose covariance structure is determined by the fundamental solution of the equation. As a concrete example, this paper describes the fundamental solution of the Helmholtz equation on the rotation group and the resulting covariance structure.

Psoriasis is a chronic inflammatory skin disease that affects over 3% of the population. Various methods are currently used to evaluate psoriasis severity and to monitor therapeutic response. The PASI system of scoring is widely used for evaluating psoriasis severity. It employs a visual analogue scale to score the thickness, redness (erythema), and scaling of psoriasis lesions. However, PASI scores are subjective and suffer from poor inter- and intra-observer concordance. As an integral part of developing a reliable evaluation method for psoriasis, an algorithm is presented for segmenting scaling in 2-D digital images. The algorithm is believed to be the first to localize scaling directly in 2-D digital images. The scaling segmentation problem is treated as a classification and parameter estimation problem. A Markov random field (MRF) is used to smooth a pixel-wise classification from a support vector machine (SVM) that utilizes a feature space derived from image color and scaling texture. The training sets for the SVM are collected directly from the image being analyzed giving the algorithm more resilience to variations in lighting and skin type. The algorithm is shown to give reliable segmentation results when evaluated with images with different lighting conditions, skin types, and psoriasis types.

Bayesian Spiking Neurons (BSNs) provide a probabilistic interpretation of how neurons can perform inference and learning. Learning in a single BSN can be formulated as an online maximum-likelihood expectation-maximisation (ML-EM) algorithm. This form of learning is quite slow. Here, an alternative to this learning algorithm, called Fast Learning (FL), is presented. The FL algorithm is shown to have acceptable convergence performance when compared to the ML-EM algorithm. Moreover, for our implementations the FL algorithm is approximately 25 times faster than the ML-EM algorithm. Although only approximate, the FL algorithm therefore makes learning in hierarchical BSN networks much more tractable.

This paper studies Brownian distributions on compact Lie groups. These are defined as the marginal distributions of Brownian processes and are intended as a natural extension of the well-known normal distributions to compact Lie groups. It is shown that this definition preserves key properties of normal distributions. In particular, Brownian distributions transform in a nice way under group operations and satisfy an extension of the central limit theorem. Brownian distributions on a compact Lie groupGbelong to one of two parametric familiesNL(g,C) andNR(g,C)-g∈GandCa positive-definite symmetric matrix. In particular, the parametergappears as a location parameter. An approach based on the extrinsic mean for estimation of the parametersgandCis studied in detail. It is shown thatgis the unique extrinsic mean for a Brownian distributionNL(g,C) orNR(g,C). Resulting estimates are proved to be consistent and asymptotically normal. While they may also be used to simultaneously estimategandC, it is seen this requires thatGbe embedded into a higher dimensional matrix Lie group. Going beyond Brownian distributions, it is shown the extrinsic mean can be used to recover the location parameter for a wider class of distributions arising more generally from Lévy processes. The compact Lie group structure places limitations on the analogy between normal distributions and Brownian distributions. This is illustrated by the study of multivariate Brownian distributions. These are introduced as Brownian distributions on some product group-e.g.,G×G. This paper describes their covariance structure and considers its transformation under group operations.

In this work, we present a 3-layer segmentation scheme for automatic erythema detection. First, a skin region is detected with a histogram-based Bayesian classifier. Next, the extracted skin image is represented in terms of melanin and hemoglobin components based on Independent Component Analysis (ICA). At last, a trained Support Vector Machine (SVM) is applied to identify erythema areas using feature attributes from hemoglobin and melanin component images. Experiment results on our database demonstrate the effectiveness of the proposed method. This work is motivated by the need of objective assessment of psoriasis treatment for study of psoriasis therapy. Distribution of abnormal redness on skin is an important sign in evaluation of psoriasis severity, but in practice it is determined subjectively by dermatologists. Our method can be used in a therapy evaluation system to assess treatment objectively and quantitatively.

Noncommutative harmonic analysis is used to solve a nonparametric estimation problem stated in terms of compound Poisson processes on compact Lie groups. This problem of decompounding is a generalization of a similar classical problem. The proposed solution is based on a characteristic function method. The treated problem is important to recent models of the physical inverse problem of multiple scattering.

Motivated by applications in multiple scattering, we study the problem of decompounding on compact Lie groups. Employing tools from harmonic analysis, we give a nonparametric approach to this problem. The case of the special orthogonal group SO(3) is discussed in detail.

The information transfer through a single neuron is a fundamental information processing in the brain. This paper studies the information-theoretic capacity of a single neuron by treating the neuron as a communication channel. Two different models are considered. The temporal coding model of a neuron as a communication channel assumes the output is ¿ where ¿ is a gamma-distributed random variable corresponding to the interspike interval, that is, the time it takes for the neuron to fire once. The rate coding model is similar; the output is the actual rate of firing over a fixed period of time. We prove that for both models, the capacity achieving distribution has only a finite number of probability mass points. This allows us to compute numerically the capacity of a neuron. Our capacity results are in a plausible range based on biological evidence to date.

Information transfer through a single neuron is a fundamental component of information processing in the brain, and computing the information channel capacity is important to understand this information processing. The problem is difficult since the capacity depends on coding, characteristics of the communication channel, and optimization over input distributions, among other issues. In this letter, we consider two models. The temporal coding model of a neuron as a communication channel assumes the output is tau where tau is a gamma-distributed random variable corresponding to the interspike interval, that is, the time it takes for the neuron to fire once. The rate coding model is similar; the output is the actual rate of firing over a fixed period of time. Theoretical studies prove that the distribution of inputs, which achieves channel capacity, is a discrete distribution with finite mass points for temporal and rate coding under a reasonable assumption. This allows us to compute numerically the capacity of a neuron. Numerical results are in a plausible range based on biological evidence to date.

The orthogonal frequency division multiplexing (OFDM) transmission scheme equipped with multiple receive antennas increases robustness against frequency-selective fading and combats loss in SNR. Based on the multichannel signaling property, this letter develops a frequency-domain blind channel estimator for OFDM systems without cyclic prefix (CP). With high data efficiency and low computational complexity, the proposed algorithm is able to identify the channels using a single received OFDM data block. Numerical simulations show that the proposed method performs satisfactorily with only one or very few received OFDM blocks, as compared to the existing subspace-based methods which require many more data records.

Single-input multiple-output (SIMO) orthogonal frequency division multiplexing (OFDM) is an appealing multi- carrier transmission technique for combating frequency-selective fading and increasing signal-to-noise ratio. To retrieve transmitted data correctly, reliable estimation of time-dispersive channels is important. This letter develops an improved cross-relation (CR) based blind SIMO-OFDM channel estimation method in the frequency domain. The significance of the proposed algorithm is twofold. First, it is highly data-efficient in that the SIMO-OFDM channels can be blindly identified using a single received data block. Second, the proposed method only requires an upper bound rather than the exact knowledge of the channel length. Simulation results show that the new method performs favorably compared to the existing CR- and subspace-based methods.

This work investigates the design of linear precoders for ARQ packet retransmissions in multi-input multi-output (MIMO) systems. We consider transmitter precoder design based on partial MIMO channel information in the form of their covariance feedback. Our objective is to maximize the ergodic mutual information provided by multiple (re)transmissions of a packet subject to transmission power constraint. We propose a set of near-optimal successive linear ARQ precoders for flat fading MIMO channels. This progressive linear ARQ precoder combines the appropriate power loading and the reverse-order pairing of singular values in the current retransmission with previous transmissions. This reverse-order pairing is a special feature unique to our sequential ARQ preceding approach with demonstrated performance gains.

This letter develops a fast Fourier transform (FFT)-based method for estimating the impulse response of FIR single-input multiple-output (SIMO) channels driven by an unknown deterministic signal. The proposed algorithm successfully handles very short data sequences, for which the existing second-order statistics based methods, e.g., the subspace (SS), cross-relation (CR), and shifted correlation (SC) algorithms, are known to suffer performance degradation due to inaccurate statistics. The new method significantly outperforms the SS, CR, and SC methods with short sequences of observation data. This proposed method is computationally efficient for achieving good performance when data sequences are inevitably short, as in certain practical applications.

With a rapidly changing channel in mobile communications, there is a scarcity of data samples, posing a challenge to reliable system identification. To address the problem, this paper presents a low complexity frequency-domain approach to blind single-input multiple-output (SIMO) system identification. The proposed approach is straightforward in concept and takes advantage of the computational power of FFT (fast Fourier transform). As a result, the new method is very efficient and effective for short data sequences, for which second-order statistics (SOS) based subspace methods suffer performance deterioration. Therefore, the proposed approach is a desirable alternative to SOS-based subspace methods to achieve good performance when data sequence is inevitably short in certain practical applications

The autoregressive (AR) process is fundamental to linear signal processing and is commonly used to model the behaviour of an object evolving on Euclidean space. In real life, there are myriad examples of objects evolving not on flat spaces but on curved spaces such as the surface of a sphere. For instance, wind-direction studies in meteorology and the estimation of relative rotations of tectonic plates based on observations on the Earth's surface deal with spherical data, while subspace tracking in signal processing is actually inference on the Grassmann manifold. This paper considers how to extend the AR process to one evolving on a curved space, or in a general, a manifold. Doing so is non-trivial, and in fact, several different extensions are proposed, along with their advantages and disadvantages. Algorithms for estimating the parameters of these generalized AR processes are derived

This paper investigates the optimal linear precoder design for packet retransmissions in multi-input-multi-output (MIMO) systems. To fully utilize the time diversity provided by automatic repeat request (ARQ), we derive a sequence of successive optimal linear ARQ precoders for flat fading MIMO channels, which minimize the mean-square error between the transmitted data and the joint receiver output. The optimization is subject to an overall transmit power constraint. This progressive linear ARQ precoder combines the appropriate power loading and the optimal pairing of channel matrix singular values in the current retransmission with previous transmissions. This optimal pairing is a special feature unique to our sequential ARQ precoding approach. Simulation results demonstrate the effectiveness of this optimized ARQ precoding in reducing symbol MSE and detection bit-error rate.

Traditionally, the majority of non-linear signal processing problems were tackled either by appropriate linearisations or by some ad hoc technique. By contrast, linear signal processing problems are routinely solved systematically by astute application of results from linear algebra. This paper shows by example how differential geometry provides the necessary tools and mindset for systematically solving certain non-linear problems commonly encountered in signal processing.

We present new algorithms for refining the estimates of the eigenvectors of a real symmetric matrix. Using techniques from calculus, we show that the algorithms converge locally cubically fast. By this we mean that locally all eigenvalue-eigenvector pairs simultaneously converge at a cubic rate. This is in contrast to well known shifted QR algorithms, which depending on the shifting strategy employed, have only one (or at most a small subset) of the eigenvalue-eigenvector pairs converging cubically at any one time. The algorithms are well suited to the situation where one needs to compute the eigenvectors of a perturbed matrix A + E based on a good estimate of the eigenvectors of a matrix A. Such a situation frequently appears in tracking applications.

The paper investigates the constrained capacity of sequential ARQ linear precoding in flat fading MIMO systems. To utilize fully the time diversity provided by ARQ, we derive the optimal linear ARQ precoders in a flat fading MIMO channel with the objective of maximizing the mutual information delivered by multiple transmissions of the same packet. The capacity-maximizing ARQ precoders combine water-pouring power loading and the optimal pairing of singular vectors in the current retransmission with those from previous transmissions. This optimal pairing is a special feature unique to sequential ARQ precoding and plays important role in maximizing the mutual information.

Motivated by applications in fuzzy control, robotics and vision, this paper considers the problem of computing the centre of mass (precisely, the Karcher mean) of a set of points defined on a compact Lie group, such as the special orthogonal group consisting of all orthogonal matrices with unit determinant. An iterative algorithm, whose derivation is based on the geometry of the problem, is proposed. It is proved to be globally convergent. Interestingly, the proof starts by showing the algorithm is actually a Riemannian gradient descent algorithm with fixed step size.

Finite alphabet optimisation problems occur in many fields of engineering, including wireless communications and blind source separation. An optimal solution through exhaustive search is often computationally intractable, so sub-optimal solutions are employed. One popular approach is simply to round each element of the unconstrained solution to the nearest member of the known alphabet. The paper presents a novel approach which has better error performance than rounding but with only a moderate increase in complexity. The method uses Fisher information to determine the order in which optimisation is carried out. The inverse of the Fisher information matrix indicates which element of the estimate is, on average, most likely to have the smallest error. Thus the first element to be optimised is the one most likely to be correct. This then improves the likelihood of subsequent elements being correct. The method is developed and an example is included of its application to the discrete blind source separation problem.

A semi-blind channel identification method is proposed for block transmission systems with guard intervals. Usually, block transmission systems append a sequence of L-1 zeros (where L is an upper bound on the channel length), called a guard interval, to each block before transmitting the blocks so as to prevent interblock interference. The paper shows that by replacing the L-1 zeros between each block with specially chosen sequences of L known symbols, a penalty of just one extra symbol per block, it is possible to determine directly the channel coefficients from L consecutively received blocks. For non-time-varying channels, the scheme is equivalent to transmitting a training sequence every L blocks, and hence it can be considered as a way of distributing a training sequence over L blocks. The benefit of the proposed method is twofold. Firstly, due to its distributed nature, it outperforms the training sequence method when the channel varies with time. Secondly, it has a lower latency delay than the traditional training sequence based method, yet achieves the same or better performance. Also, the proposed method retains all the advantages of guard intervals.

A new computationally simple approach to blind decoding of orthogonal space-time block codes (STBC) is proposed. Our approach estimates the channel matrix in a closed form and uses this estimate in the maximum likelihood (ML) receiver to decode the symbols. It exploits specific properties of the orthogonal STBC and is free of major drawbacks of other blind space-time decoding schemes.

A new computationally simple approach to blind decoding of orthogonal space-time block codes is proposed. Our approach estimates the channel matrix in a closed form and uses this estimate in the maximum likelihood (ML) receiver to decode the symbols. It exploits specific properties of the orthogonal space-time block codes (STBCs) and is free of most of the shortcomings of other blind space-time decoding schemes.

We present a new method for identification of FIR MIMO channels driven by unknown, uncorrelated and colored sources. This method, belonging to the BID (blind identification by decorrelation) family, makes use of the mutual uncorrelation of the unknown sources by first decorrelating the observed signals into two uncorrelated groups. The two decorrelators are then used to estimate the channel matrix (i.e., MIMO channel transfer function matrix) up to a constant matrix. This constant matrix is finally determined using a BID method for instantaneous MIMO channels. This new method, named BID-G, is shown to be much more robust than the subspace method that requires the channel matrix to be irreducible and column-reduced.

Blind channel identification techniques usually exploit a known property of the source symbols such as a statistical or finite alphabet property. Recently, a purely algebraic approach that relies on guard intervals (sequences of zeros equal or longer in length than the channel memory) inserted between transmitted blocks has been considered. It was proved that only two received blocks suffice for channel identification and source recovery. We approach the channel identification problem from a z-domain perspective. It is shown that, in the z-domain, the channel is a common factor in all received blocks (this fundamental property appears to have gone unnoticed in the literature). This allows a subspace method for computing the greatest common divisor (GCD) to be applied to the channel estimation problem. The algorithm achieves the theoretical limit in that only two received blocks are required before the channel can be identified, but of course, the more blocks that are used, the better the performance in the presence of noise.

This paper analyses the fast subspace tracking algorithm recently proposed by Attallah and Abed-Meraim. Specifically, the associated Ordinary Differential Equation is derived and its properties studied in detail. Partial comparisons with other subspace tracking algorithms are made.

This paper presents a novel approach to the unitary joint diagonalisation problem. We approach the problem as an optimisation problem over the manifold of unitary matrices and obtain steepest descent and Newton algorithms. Both algorithms converge to the same solution as the well known Joint Approximate Diagonalisation of Eigen-matrices (JADE) algorithm. However, our Newton algorithm achieves a quadratic convergence rate almost immediately if suitably initialised, whereas JADE can (in some cases) take up to 30 sweeps before it starts converging to the solution at a quadratic rate. Simulations illustrate that our steepest descent algorithm can be used to find a suitable starting point for our Newton algorithm which then refines our estimate at a quadratic rate. This two stage algorithm requires less iterations overall than JADE, with the cost per iteration being the same as JADE. Our methods, therefore, offer a novel alternative to JADE.

An error correcting code concatenated with a linear precoder is known to be a simple yet effective coding strategy for transmitting blocks of data over frequency selective fading channels. Although optimal maximum-likelihood decoding of such combined codes is prohibitively expensive to perform in general, this paper demonstrates that a sub-class of combined codes can be decoded both optimally and with an acceptable amount of computational complexity. The codes in this sub-class are called M-Codes because they can be implemented by a linear precoder followed by an element-wise modulo operation. The decoding algorithm is a modification of Viterbi's algorithm, the main difference being that the number of states varies according to the depth.

This paper discusses the problem of blind identification of MIMO convolutive channels when the sources are quasi-stationary. No other assumptions are made about the sources; i.e., they can be temporally white or colored and they can assume any distributions. We show that by using second order statistics of the channel outputs, under some mild conditions on the non-stationarity of sources and that the channel is column-wise coprime, identification can be achieved up to a scalar ambiguity and column permutation of the original MIMO channel. We also present an efficient, two step frequency domain algorithm for identifying the channel. To demonstrate the performance of the new algorithm numerical simulations are presented.

Certain linear and affine precoders introduce enough algebraic redundancy to enable the receiver to identify a. single-input single-output finite-impulse response channel without making any statistical assumptions on the source sequence. However, quite surprisingly, the traditional steepest descent least squares algorithm for estimating the channel often fails to converge, even in the absence of noise. This article explains why this is the case and derives a novel steepest descent algorithm on complex projective space that is guaranteed to converge. The complex projective space formulation also provides a standard framework for understanding different performance measures proposed in the literature.

Channels with spectral nulls are sometimes dubbed bad channels because they can cause poor performance in communication systems. This article investigates the validity of this intuition by studying the geometry of an orthogonal frequency-division multiplex (OFDM) system. It is shown that the subchannel attenuation coefficients form a natural coordinate system for describing finite-impulse response (FIR) channels in an OFDM framework. It is also shown that channels with spectral nulls are geometrically significant; they form the faces of the convex set of all subchannel attenuation coefficients. This novel perspective makes it immediately clear why the worst performance of a linearly precoded OFDM system is achieved over a channel having the greatest number of spectral nulls. The practical implications of these results are discussed.

A modification of Moulines' blind second order statistical channel subspace approach (see Moulines, E. et al., IEEE Trans. on Sig. Proc., vol.43, no.2, p.516-25, 1995) is proposed for the identification of single input multiple output finite impulse response channels. The modification exploits the transmitter redundancy introduced by a trailing zero filter bank precoder. The method is shown to be robust to common zeros and channel order over-estimation errors.

If two wide-sense stationary time series are correlated then one can be used to predict the other. The reduced rank Wiener filter is the rank-constrained linear operator which maps the current value of one time series to an estimate of the current value of the other time series in an optimal way. A closed-form solution exists for the reduced rank Wiener filter. This paper studies the problem of determining the reduced rank FIR filter which optimally predicts one time series given the other. This optimal FIR filter is called the convolutive reduced rank Wiener filter, and it is proved that determining it is equivalent to solving a weighted low rank approximation problem. In certain cases a closed-form solution exists, and in general, the iterative optimisation algorithm derived here can be used to converge to a locally optimal convolutive reduced rank Wiener filter.

In high speed wireless multimedia communications, it is necessary to take measures to mitigate the adverse effects of frequency selective fading channels. If the multimedia communications is packet based, and if each packet is suitably linearly or affinely precoded prior to its transmission, then it is possible to identify the channel on a packet-by-packet basis. Somewhat surprisingly though, this paper shows that the standard iterative least squares algorithm for identifying the channel often fails to converge. It then goes on to derive a novel channel identification algorithm which is guaranteed to converge.

This paper proposes a channel coding scheme for multimedia communications over unknown multipath channels. The key feature of this scheme is that it allows the receiver to estimate the source symbols with the same accuracy regardless of the shape of the channel spectrum. This, together with its computational simplicity, makes it particularly attractive for low cost wireless multimedia applications.

It is known that precoding a signal prior to its transmission through an unknown finite impulse response channel facilitates the equalisation of the channel. Moreover, since linear precoders spread the spectrum, the equalisation process mitigates the effects of channel spectral nulls caused by frequency selective fading. This paper argues that certain affine precoders are more efficient than linear precoders. Here, the efficiency of a precoder is a measure of its ability to enable the receiver to recover the source signal relative to the amount of redundancy introduced by the precoder. An efficient affine precoder is heuristically derived and simulations used to demonstrate that it is superior to both the traditional training sequence method and the filter bank precoding scheme. This precoding and equalisation scheme naturally extends to time varying channels.

This letter studies the problem of identifying a single-input single-output channel by linearly precoding the input. Although it is known that filterbanks induce cyclostationarity if the input is stationary, this work demonstrates that filterbanks induce a special form of spectral redundancy even if the input is deterministic. This frequency domain interpretation of filterbanks nicely characterizes all linear precoders. It not only helps to identify the strengths and aid in the design of linear precoders, it leads to frequency domain subspace methods for the identification of AR, MA and ARMA channels.

It is a standard result that a finite impulse response channel of length L can be uniquely identified by feeding in a known (and persistently exciting) sequence of 2L-1 consecutive data points. Equivalently, given only 2L-2 consecutive data points, the channel can be uniquely identified up to a multiplicative constant. This paper significantly extends the identifiability criterion to the case when the known inputs are non-consecutively located. It is argued that by introducing 2L-1 non-consecutively spaced zeros into the input stream, for almost all input sequences, the channel can be uniquely identified up to a multiplicative constant. Furthermore, the result can be extended to the case when the known inputs are non-zero, in which case the channel can almost always be identified uniquely. To arrive at these results, general properties of systems of polynomial equations are derived. These properties do not seem to have appeared in the literature before.

A discrete time Poisson process whose rate evolves as the square of the state of a linear Gaussian dynamical system is studied. An optimal filter is derived, yielding real-time estimates of the Poisson rate. Also a suboptimal filter based on an Edgeworth series expansion is derived.