In this paper, we describe a set of robust algorithms for group-wise registration using both rigid and non-rigid transformations of multiple unlabelled point-sets with no bias toward a given set. These methods mitigate the need to establish a correspondence among the point-sets by representing them as probability density functions where the registration is treated as a multiple distribution alignment. Holder's and Jensen's inequalities provide a notion of similarity/distance among point-sets and Rényi's second order entropy yields a closed-form solution to the cost function and update equations. We also show that the methods can be improved by normalizing the entropy with a scale factor. These provide simple, fast and accurate algorithms to compute the spatial transformation function needed to register multiple point-sets. The algorithms are compared against two well-known methods for group-wise point-set registration. The results show an improvement in both accuracy and computational complexity.

This paper proposes a deterministic explanation for mutual-information-based image registration (MI registration). The explanation is that MI registration works because it aligns certain image partitions. This notion of aligning partitions is new, and is shown to be related to Schur- and quasi-convexity. The partition-alignment theory of this paper goes beyond explaining mutual- information. It suggests other objective functions for registering images. Some of these newer objective functions are not entropy-based. Simulations with noisy images show that the newer objective functions work well for registration, lending support to the theory. The theory proposed in this paper opens a number of directions for further research in image registration. These directions are also discussed.

Information theory provides principled ways to analyze different inference and learning problems, such as hypothesis testing, clustering, dimensionality reduction, classification, and so forth. However, the use of information theoretic quantities as test statistics, that is, as quantities obtained from empirical data, poses a challenging estimation problem that often leads to strong simplifications, such as Gaussian models, or the use of plug in density estimators that are restricted to certain representation of the data. In this paper, a framework to nonparametrically obtain measures of entropy directly from data using operators in reproducing kernel Hilbert spaces defined by infinitely divisible kernels is presented. The entropy functionals, which bear resemblance with quantum entropies, are defined on positive definite matrices and satisfy similar axioms to those of Renyi's definition of entropy. Convergence of the proposed estimators follows from concentration results on the difference between the ordered spectrum of the Gram matrices and the integral operators associated to the population quantities. In this way, capitalizing on both the axiomatic definition of entropy and on the representation power of positive definite kernels, the proposed measure of entropy avoids the estimation of the probability distribution underlying the data. Moreover, estimators of kernel-based conditional entropy and mutual information are also defined. Numerical experiments on independence tests compare favorably with state-of-the-art.

Exploratory tools that are sensitive to arbitrary statistical variations in spike train observations open up the possibility of novel neuroscientific discoveries. Developing such tools, however, is difficult due to the lack of Euclidean structure of the spike train space, and an experimenter usually prefers simpler tools that capture only limited statistical features of the spike train, such as mean spike count or mean firing rate. We explore strictly positive-definite kernels on the space of spike trains to offer both a structural representation of this space and a platform for developing statistical measures that explore features beyond count or rate. We apply these kernels to construct measures of divergence between two point processes and use them for hypothesis testing, that is, to observe if two sets of spike trains originate from the same underlying probability law. Although there exist positive-definite spike train kernels in the literature, we establish that these kernels are not strictly definite and thus do not induce measures of divergence. We discuss the properties of both of these existing nonstrict kernels and the novel strict kernels in terms of their computational complexity, choice of free parameters, and performance on both synthetic and real data through kernel principal component analysis and hypothesis testing.

This paper addresses the estimation of symmetric χ2-divergence between two point processes. We propose a novel approach by, first, mapping the space of spike trains in an appropriate functional space, and then, estimating the divergence in this functional space using a least square regression approach. We compare the proposed approach with other available methods on simulated data, and discuss its pros and cons.

In wireless multi-hop packet radio networks (MPRNs) that employ geographic transmissions, sleep schedules or node activation techniques may be used to power off some nodes to conserve energy. We consider the problem of selecting which nodes should power on to listen to a scheduled transmission when the channel suffers from random fading. We choose the objective of maximizing the expected value of the distance covered in a single transmission between a transmitter and the farthest receiver that successfully receives the packet, under a constraint on the expected number of receivers that turn on. Since there is a tradeoff between the distance of a node from the transmitter and the probability that the node receives the message correctly, we propose to use node-activation based on link-distance (NA-BOLD). We investigate optimal and sub-optimal NA-BOLD schemes and compare their performance with that of schemes that use a constant sleep schedule for every node within some radius of the transmitter. Our results show that the proposed NA-BOLD schemes achieve significantly larger transmission distances than conventional schemes.

In mobile ad hoc networks (MANETs), conventional packet forwarding schemes that pre-select the next-hop receivers for a packet may fail if the channel coherence time is on the order of the typical packet duration because the pre-selected node may often suffer a deep fade for the duration of the packet. An alternative approach is geographic transmission, in which the packet is transmitted in the direction of the destination, but the next-hop forwarding node is selected among those nodes that are in the direction of the destination and that correctly recover the message. This approach takes advantage of multi-user diversity to significantly improve the probability of the packet being correctly received by a forwarding agent. However, this approach places additional burden on the energies of the mobile nodes if the forwarding scheme requires all of the next-hop neighbors of the transmitter (that are in the direction of the destination) attempt to receive a transmitted message. In this paper, we consider the joint design of node-activation strategies and transmission rates to maximize the expected value of transport capacity over a Nakagami-m channel under a constraint on the expected number of nodes that attempt to receive a packet. We show that our approach offers better performance than other approaches.

We consider a collection of battery-operated, low-power sensors randomly deployed within a geographic region for the purpose of sensing/monitoring the environment. In such a scenario, it is desirable to allow nodes to sleep (power off) whenever possible to conserve energy and increase the lifetime of the network. However, when the channel is subject to random variations, such as fading, there is an inherent trade-off between node activation and diversity. We consider a scenario in which a fixed number of nodes are allowed to turn on around a transmitter and the goal is to maximize the expected value of the distance that the message travels from the transmitter to the farthest receiver. We investigate the design of an algorithm to probabilistically determine whether a node should turn on based on its distance from the transmitter. This turns out to be a difficult probabilistic problem. We transform this problem into an analytical problem that provides a sub-optimal solution to the probabilistic problem. We compare the performance of this approach to a simpler protocol that turns on all the nodes within some radius of the transmitter. The results show that the link-distance based activation protocol provides significant gains in maximizing the expected value of the transmission distance to the farthest receiver, compared to the simpler protocol.

Image retrieval is a common problem in many computer vision applications and the literature abounds with techniques with impressive retrieval accuracies. Several of these techniques use probability distributions to represent the "objects" they intend to retrieve. We present a novel approach to search such collections of distributions efficiently. Like many standard data structures, our method uses an "average" to represent a large set (Le., cluster) of objects, thus allowing the search to disregard an unpromising subset with only one comparison to its average. Our contribution lies in choosing the average: Inspired by information theory, we choose a representative that is optimally "close" in a minimax sense to all members of its set when measured using the Kullback-Liebler (KL) divergence. We present a texture retrieval system and test it on the CUReT database, measuring accuracy and efficiency. We find that using the KL-center yields speed ups of more than a factor of three over an exhaustive search while guaranteeing identical accuracy. The KL-center also out-performs other commonly used representatives such as the arithmetic mean. Although we present results only in texture retrieval, our approach will likely aid image and shape retrieval as well.

In this paper, we use the cumulative distribution of a random variable to define its information content and thereby develop an alternative measure of uncertainty that extends Shannon entropy to random variables with continuous distributions. We call this measure cumulative residual entropy (CRE). The salient features of CRE are as follows: 1) it is more general than the Shannon entropy in that its definition is valid in the continuous and discrete domains, 2) it possesses more general mathematical properties than the Shannon entropy, and 3) it can be easily computed from sample data and these computations asymptotically converge to the true values. The properties of CRE and a precise formula relating CRE and Shannon entropy are given in the paper. Finally, we present some applications of CRE to reliability engineering and computer vision.

In this note we present a coupled optimization model for boundary determination. One part of the model incorporates a prior shape into a geometric active contour model with a fixed parameter. The second part determines the 'best' parameter used in the first part by maximizing the mutual information of the image geometry between the prior and an aligned novel image over all the alignments that are the solutions of the first part corresponding to different parameters. We also present an alternative method, which generates an intensity model formed as the average of a set of aligned training images. Experimental results on cardiac ultrasound images are presented. These results indicate that the proposed model provides close agreement with expert traced borders, and the parameter determined in this model for one image can be used for images with similar properties. The existence of a solution to the proposed minimization problem is also discussed.

We use the cumulative distribution of a random variable to define the information content in it and use it to develop a novel measure of information that parallels Shannon entropy, which we dub cumulative residual entropy (CRE). The key features of CRE may be summarized as, (1) its definition is valid in both the continuous and discrete domains, (2) it is mathematically more general than the Shannon entropy and (3) its computation from sample data is easy and these computations converge asymptotically to the true values. We define the cross-CRE (CCRE) between two random variables and apply it to solve the uni- and multimodal image alignment problem for parameterized (rigid, affine and projective) transformations. The key strengths of the CCRE over using the now popular mutual information method (based on Shannon's entropy) are that the former has significantly larger noise immunity and a much larger convergence range over the field of parameterized transformations. These strengths of CCRE are demonstrated via experiments on synthesized and real image data.

In this paper, two iterative methods for correcting attenuation errors in positron emission tomography (PET) are proposed. The first method is a maximum likelihood (ML) method that simultaneously estimates the emission intensity and attenuation density. The second one is a least squares (LS) reprojection method, where the survival probabilities are estimated by reprojecting a LS estimate of the attenuation density. Both methods are guaranteed to produce nonnegative estimates of the emission intensity and attenuation density, provided that the initial estimates are nonnegative. From simulation studies, it appears that the proposed methods have less noise and provide higher quality images, particularly for short transmission scans (less than 10 minutes), than the EMML algorithm with standard correction.

We present unpenalized and penalized weighted least-squares (WLS) reconstruction methods for positron emission tomography (PET), where the weights are based on the covariance of a model error and depend on the unknown parameters. The penalty function for the latter method is chosen so that certain a priori information is incorporated. The algorithms used to minimize the WLS objective functions guarantee nonnegative estimates and, experimentally, they converged faster than the maximum likelihood expectation-maximization (ML-EM) algorithm and produced images that had significantly better resolution and contrast. Although simulations suggest that the proposed algorithms are globally convergent, a proof of convergence has not yet been found. Nevertheless, we are able to show that the unpenalized method produces estimates that decrease the objective function monotonically with increasing iterations.